The MySQL server is running with the --read-only option so it cannot execute this statement

[Andrew McLeod]

Similar to https://bugs.launchpad.net/charm-mysql-innodb-cluster/+bug/1882205

s390x groovy deployment, nova-cloud-controller nova-scheduler.log:

https://pastebin.canonical.com/p/6BSnSF9xf9/

Tl;DR

2020-10-29 10:15:16.616 1943 WARNING nova.scheduler.client.report [req-fe524f1c-8b05-491e-87e8-2d559d2153cd 6784c34ab0644cb0a30f9d1c503159ec 3fd1b5542dbd411084de9dc21a66389f - b02eeade72c0480c9fbe8195849f3ab6 b02eeade72c0480c9fbe8195849f3ab6] Failed to save allocation for c3a8f21b-cdf9-49cc-8c27-60b5c5bb9a09. Got HTTP 500: {"errors": [{"status": 500, "title": "Internal Server Error", "detail": "The server has either erred or is incapable of performing the requested operation.\n\n (pymysql.err.InternalError) (1290, 'The MySQL server is running with the --read-only option so it cannot execute this statement') [SQL: INSERT INTO consumers (created_at, updated_at, uuid, project_id, user_id, generation) VALUES (%(created_at)s, %(updated_at)s, %(uuid)s, %(project_id)s, %(user_id)s, %(generation)s)] [parameters: {'created_at': datetime.datetime(2020, 10, 29, 10, 15, 16, 611378), 'updated_at': None, 'uuid': 'c3a8f21b-cdf9-49cc-8c27-60b5c5bb9a09', 'project_id': 1, 'user_id': 1, 'generation': 0}] (Background on this error at: http://sqlalche.me/e/13/2j85) ", "code": "placement.undefined_code", "request_id": "req-a3eb7806-442f-4ab0-b8f5-6befd7920cb7"}]}

juju status and cluster-status both look OK:

(tempest) jenkins@mosci-s390x-jslave-persist-21:~/tools/openstack-charm-testing/tempest$ juju run-action --wait mysql/0 cluster-status
unit-mysql-0:
  UnitId: mysql/0
  id: "107"
  results:
    cluster-status: '{"clusterName": "jujuCluster", "defaultReplicaSet": {"name":
      "default", "primary": "10.13.3.196:3306", "ssl": "REQUIRED", "status": "OK",
      "statusText": "Cluster is ONLINE and can tolerate up to ONE failure.", "topology":
      {"10.13.3.196:3306": {"address": "10.13.3.196:3306", "mode": "R/W", "readReplicas":
      {}, "replicationLag": null, "role": "HA", "status": "ONLINE", "version": "8.0.22"},
      "10.13.3.223:3306": {"address": "10.13.3.223:3306", "mode": "R/O", "readReplicas":
      {}, "replicationLag": null, "role": "HA", "status": "ONLINE", "version": "8.0.22"},
      "10.13.3.238:3306": {"address": "10.13.3.238:3306", "mode": "R/O", "readReplicas":
      {}, "replicationLag": null, "role": "HA", "status": "ONLINE", "version": "8.0.22"}},
      "topologyMode": "Single-Primary"}, "groupInformationSourceMember": "10.13.3.196:3306"}'
  status: completed
  timing:
    completed: 2020-10-29 10:36:03 +0000 UTC
    enqueued: 2020-10-29 10:35:59 +0000 UTC
    started: 2020-10-29 10:36:00 +0000 UTC
(tempest) jenkins@mosci-s390x-jslave-persist-21:~/tools/openstack-charm-testing/tempest$ juju status mysql
Model Controller Cloud/Region Version SLA Timestamp
s390x-mosci-s390x s390x-mosci-s390x manual 2.8.5 unsupported 10:36:16Z

App Version Status Scale Charm Store Rev OS Notes
mysql 8.0.22 active 3 mysql-innodb-cluster jujucharms 51 ubuntu

Unit Workload Agent Machine Public address Ports Message
mysql/0 active idle 0/lxd/2 10.13.3.238 Unit is ready: Mode: R/O
mysql/1* active idle 1/lxd/3 10.13.3.223 Unit is ready: Mode: R/O
mysql/2 active idle 2/lxd/3 10.13.3.196 Unit is ready: Mode: R/W

[Andrew McLeod (admcleod)]

I tried manually pointing the db connection string in nova.conf to the R/W host, but still got the same error.

[Alex Kavanagh (ajkavanagh)]

Setting to incomplete. This bug was opened approx 2.5 years ago and hasn't been added to since. It will time out in around 2 months if no further information is added.

[Launchpad Janitor (janitor)]

[Expired for MySQL InnoDB Cluster Charm because there has been no activity for 60 days.]