mysql-innodb-cluster - Gets in error with "hook failed: "cluster-relation-changed""
Ussuri Bundle - cs:bundle/openstack-base-69
mysql-innodb-cluster charms used - cs:mysql-innodb-cluster-0 and cs:~openstack-charmers-next/mysql-innodb-cluster-42
Debug logs -
unit-mysql-innodb-cluster-6: 11:17:12 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: hooks/relations/mysql-innodb-cluster/peers.py:69:joined:cluster
unit-mysql-innodb-cluster-6: 11:17:12 DEBUG unit.mysql-innodb-cluster/6.juju-log cluster:90: tracer>
tracer: set flag cluster.available
tracer: ++ queue handler reactive/mysql_innodb_cluster_handlers.py:88:create_remote_cluster_user
tracer: -- dequeue handler reactive/mysql_innodb_cluster_handlers.py:66:send_cluster_connection_info
unit-mysql-innodb-cluster-6: 11:17:12 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: hooks/relations/mysql-innodb-cluster/peers.py:75:changed:cluster
unit-mysql-innodb-cluster-6: 11:17:12 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: hooks/relations/tls-certificates/requires.py:109:broken:certificates
unit-mysql-innodb-cluster-6: 11:17:13 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: hooks/relations/mysql-router/provides.py:47:joined:db-router
unit-mysql-innodb-cluster-6: 11:17:13 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: hooks/relations/mysql-router/provides.py:53:changed:db-router
unit-mysql-innodb-cluster-6: 11:17:13 INFO unit.mysql-innodb-cluster/6.juju-log cluster:90: Invoking reactive handler: reactive/mysql_innodb_cluster_handlers.py:88:create_remote_cluster_user
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.juju-log cluster:90: Creating remote users.
unit-mysql-innodb-cluster-6: 11:17:14 WARNING unit.mysql-innodb-cluster/6.juju-log cluster:90: This instance is not yet clustered: cannot determine the cluster status.
unit-mysql-innodb-cluster-6: 11:17:14 WARNING unit.mysql-innodb-cluster/6.juju-log cluster:90: Cannot determine the cluster primary RW node for writes.
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.juju-log cluster:90: Opening db connection for root@None
unit-mysql-innodb-cluster-6: 11:17:14 WARNING unit.mysql-innodb-cluster/6.juju-log cluster:90: User clusteruser already exists.
unit-mysql-innodb-cluster-6: 11:17:14 ERROR unit.mysql-innodb-cluster/6.juju-log cluster:90: Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/charm/reactive/mysql_innodb_cluster_handlers.py", line 103, in create_remote_cluster_user
instance.create_cluster_user(
File "lib/charm/openstack/mysql_innodb_cluster.py", line 519, in create_cluster_user
m_helper.execute(SQL_CLUSTER_USER_GRANT.format(
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charmhelpers/contrib/database/mysql.py", line 178, in execute
cursor.execute(sql)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1290, 'The MySQL server is running with the --super-read-only option so it cannot execute this statement')
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed Traceback (most recent call last):
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/charm/hooks/cluster-relation-changed", line 22, in <module>
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed main()
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed bus.dispatch(restricted=restricted_mode)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed _invoke(other_handlers)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed handler.invoke()
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed self._action(*args)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/charm/reactive/mysql_innodb_cluster_handlers.py", line 103, in create_remote_cluster_user
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed instance.create_cluster_user(
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "lib/charm/openstack/mysql_innodb_cluster.py", line 519, in create_cluster_user
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed m_helper.execute(SQL_CLUSTER_USER_GRANT.format(
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/charmhelpers/contrib/database/mysql.py", line 178, in execute
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed cursor.execute(sql)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed res = self._query(query)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed db.query(q)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed File "/var/lib/juju/agents/unit-mysql-innodb-cluster-6/.venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed _mysql.connection.query(self, query)
unit-mysql-innodb-cluster-6: 11:17:14 DEBUG unit.mysql-innodb-cluster/6.cluster-relation-changed MySQLdb._exceptions.OperationalError: (1290, 'The MySQL server is running with the --super-read-only option so it cannot execute this statement')
unit-mysql-innodb-cluster-6: 11:17:14 ERROR juju.worker.uniter.operation hook "cluster-relation-changed" failed: exit status 1
Hit this issue while deploying "Ussuri" on focal.
mysql-innodb- cluster - Gets in error with "hook failed: "cluster- relation- changed" " openstack- base-69 cluster charms used - cs:mysql- innodb- cluster- 0 and cs:~openstack- charmers- next/mysql- innodb- cluster- 42
Ussuri Bundle - cs:bundle/
mysql-innodb-
Debug logs - innodb- cluster- 6: 11:17:12 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: hooks/relations /mysql- innodb- cluster/ peers.py: 69:joined: cluster innodb- cluster- 6: 11:17:12 DEBUG unit.mysql- innodb- cluster/ 6.juju- log cluster:90: tracer> mysql_innodb_ cluster_ handlers. py:88:create_ remote_ cluster_ user mysql_innodb_ cluster_ handlers. py:66:send_ cluster_ connection_ info innodb- cluster- 6: 11:17:12 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: hooks/relations /mysql- innodb- cluster/ peers.py: 75:changed: cluster innodb- cluster- 6: 11:17:12 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: hooks/relations /tls-certificat es/requires. py:109: broken: certificates innodb- cluster- 6: 11:17:13 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: hooks/relations /mysql- router/ provides. py:47:joined: db-router innodb- cluster- 6: 11:17:13 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: hooks/relations /mysql- router/ provides. py:53:changed: db-router innodb- cluster- 6: 11:17:13 INFO unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Invoking reactive handler: reactive/ mysql_innodb_ cluster_ handlers. py:88:create_ remote_ cluster_ user innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Creating remote users. innodb- cluster- 6: 11:17:14 WARNING unit.mysql- innodb- cluster/ 6.juju- log cluster:90: This instance is not yet clustered: cannot determine the cluster status. innodb- cluster- 6: 11:17:14 WARNING unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Cannot determine the cluster primary RW node for writes. innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Opening db connection for root@None innodb- cluster- 6: 11:17:14 WARNING unit.mysql- innodb- cluster/ 6.juju- log cluster:90: User clusteruser already exists. innodb- cluster- 6: 11:17:14 ERROR unit.mysql- innodb- cluster/ 6.juju- log cluster:90: Hook error: juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ __init_ _.py", line 74, in main dispatch( restricted= restricted_ mode) juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 390, in dispatch other_handlers) juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 359, in _invoke invoke( ) juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 181, in invoke _action( *args) juju/agents/ unit-mysql- innodb- cluster- 6/charm/ reactive/ mysql_innodb_ cluster_ handlers. py", line 103, in create_ remote_ cluster_ user create_ cluster_ user( openstack/ mysql_innodb_ cluster. py", line 519, in create_cluster_user execute( SQL_CLUSTER_ USER_GRANT. format( juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charmhelpers/ contrib/ database/ mysql.py" , line 178, in execute execute( sql) juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ cursors. py", line 206, in execute juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ cursors. py", line 319, in _query juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ connections. py", line 259, in query connection. query(self, query) _exceptions. OperationalErro r: (1290, 'The MySQL server is running with the --super-read-only option so it cannot execute this statement')
unit-mysql-
unit-mysql-
tracer: set flag cluster.available
tracer: ++ queue handler reactive/
tracer: -- dequeue handler reactive/
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
Traceback (most recent call last):
File "/var/lib/
bus.
File "/var/lib/
_invoke(
File "/var/lib/
handler.
File "/var/lib/
self.
File "/var/lib/
instance.
File "lib/charm/
m_helper.
File "/var/lib/
cursor.
File "/var/lib/
res = self._query(query)
File "/var/lib/
db.query(q)
File "/var/lib/
_mysql.
MySQLdb.
unit-mysql- innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed Traceback (most recent call last): innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/charm/ hooks/cluster- relation- changed" , line 22, in <module> innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed main() innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ __init_ _.py", line 74, in main innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed bus.dispatch( restricted= restricted_ mode) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 390, in dispatch innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed _invoke( other_handlers) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 359, in _invoke innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed handler.invoke() innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charms/ reactive/ bus.py" , line 181, in invoke innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed self._action(*args) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/charm/ reactive/ mysql_innodb_ cluster_ handlers. py", line 103, in create_ remote_ cluster_ user innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed instance. create_ cluster_ user( innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "lib/charm/ openstack/ mysql_innodb_ cluster. py", line 519, in create_cluster_user innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed m_helper. execute( SQL_CLUSTER_ USER_GRANT. format( innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ charmhelpers/ contrib/ database/ mysql.py" , line 178, in execute innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed cursor.execute(sql) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ cursors. py", line 206, in execute innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed res = self._query(query) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ cursors. py", line 319, in _query innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed db.query(q) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed File "/var/lib/ juju/agents/ unit-mysql- innodb- cluster- 6/.venv/ lib/python3. 8/site- packages/ MySQLdb/ connections. py", line 259, in query innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed _mysql. connection. query(self, query) innodb- cluster- 6: 11:17:14 DEBUG unit.mysql- innodb- cluster/ 6.cluster- relation- changed MySQLdb. _exceptions. OperationalErro r: (1290, 'The MySQL server is running with the --super-read-only option so it cannot execute this statement') innodb- cluster- 6: 11:17:14 ERROR juju.worker. uniter. operation hook "cluster- relation- changed" failed: exit status 1
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-
unit-mysql-