Failing to cluster "This function is not available through a session to an instance already in an InnoDB cluster"

[Peter Sabaini]

I've got a test run where mysql-innodb-cluster failed to cluster:

mysql-innodb-cluster/0* waiting idle 0 172.17.112.16 Not all instances configured for clustering
mysql-innodb-cluster/1 waiting idle 1 172.17.112.9 Not all instances configured for clustering
mysql-innodb-cluster/2 waiting idle 2 172.17.112.4 Instance not yet configured for clustering

Checking the logs I can see a mysql server restart:

2020-10-28 20:44:14 INFO juju-log cluster:3: Adding instance, 172.17.112.9, to the cluster.
2020-10-28 20:44:32 DEBUG juju-log cluster:3: Instance Clustered 172.17.112.9:

* Waiting for server restart... \
* Waiting for server restart... |
* Waiting for server restart... /
* Waiting for server restart... -
* Waiting for server restart... \
* Waiting for server restart... |
* Waiting for server restart... /
* Waiting for server restart... -
* Waiting for server restart... \
* Waiting for server restart... ready

But when adding a nother instance server conn seems to break:

2020-10-28 20:44:45 ERROR juju-log cluster:3: Failed configuring instance 172.17.112.4: Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory
Configuring MySQL instance at 172.17.112.4:3306 for use in an InnoDB cluster...
This instance reports its own address as 172.17.112.4:3306
The instance '172.17.112.4:3306' is valid to be used in an InnoDB cluster.
The instance '172.17.112.4:3306' is already ready to be used in an InnoDB cluster.
Traceback (most recent call last):
  File "<string>", line 3, in <module>
mysqlsh.DBError: MySQL Error (2006): Session.run_sql: MySQL server has gone away

Then a shell fault complaining about this instance already in a cluster, afaict:

2020-10-28 20:45:14 DEBUG juju-log cluster:3: tracer: hooks phase, 0 handlers queued
2020-10-28 20:45:14 DEBUG juju-log cluster:3: tracer>
tracer: main dispatch loop, 4 handlers queued
tracer: ++ queue handler hooks/relations/mysql-innodb-cluster/peers.py:69:joined:cluster
tracer: ++ queue handler hooks/relations/mysql-innodb-cluster/peers.py:75:changed:cluster
tracer: ++ queue handler hooks/relations/tls-certificates/requires.py:109:broken:certificates
tracer: ++ queue handler reactive/mysql_innodb_cluster_handlers.py:134:configure_instances_for_clustering
2020-10-28 20:45:14 INFO juju-log cluster:3: Invoking reactive handler: reactive/mysql_innodb_cluster_handlers.py:134:configure_instances_for_clustering
2020-10-28 20:45:14 DEBUG juju-log cluster:3: Configuring instances for clustering.
2020-10-28 20:45:14 WARNING juju-log cluster:3: Instance: 172.17.112.9, already configured.
2020-10-28 20:45:14 WARNING juju-log cluster:3: Instance: 172.17.112.9, already clustered.
2020-10-28 20:45:14 INFO juju-log cluster:3: Configuring instance for clustering: 172.17.112.4.
2020-10-28 20:45:15 ERROR juju-log cluster:3: Failed configuring instance 172.17.112.4: Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: MYSQLSH (51305): Dba.configure_instance: This function is not available through a session to an instance already in an InnoDB cluster

2020-10-28 20:45:15 WARNING juju-log cluster:3: Instance: 172.17.112.4, already clustered.

Diagnostics available here: https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_func_full/openstack/charm-rabbitmq-server/757955/5/7260/index.html

[David Ames (thedac)]

This is a duplicate of LP Bug #1901771

mysqlsh.DBError: MySQL Error (2006): Session.run_sql: MySQL server has gone away

[0] https://bugs.launchpad.net/snap-mysql-shell/+bug/1901771