MySQL InnoDB Cluster not healthy: None

[macchese]

openstack-base #70

problm: after rebooting all the nodes, MySQL InnoDB Cluster remains blocked unefinitely.

Unit Workload Agent Machine Public address Ports Message
mysql-innodb-cluster/0* blocked executing 0/lxd/4 172.17.3.96 MySQL InnoDB Cluster not healthy: None
mysql-innodb-cluster/1 blocked idle 1/lxd/4 172.17.3.79 MySQL InnoDB Cluster not healthy: None
mysql-innodb-cluster/2 blocked idle 2/lxd/4 172.17.3.111 MySQL InnoDB Cluster not healthy: None

looking into /var/log/mysql/error.log of the 3 mysql units, I notice the error:

[ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error on opening a connection to 172.17.3.79:33061 on local port: 33061.
[ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error on opening a connection to 172.17.3.111:33061 on local port: 33061.'
[ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error connecting to all peers. Member join failed. Local port: 33061'
[ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] The member was unable to join the group. Local port: 33061'

Every mysqld can't contact his peers.

so I ssh into a node (172.17.3.79) but there is no mysql on port 33061, only on 33060.
ubuntu@juju-796143-1-lxd-4:~$ sudo netstat -tulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 juju-796143-1-lxd:mysql 0.0.0.0:* LISTEN 6613/mysqld
tcp 0 0 localhost:domain 0.0.0.0:* LISTEN 178/systemd-resolve
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN 347/sshd: /usr/sbin
tcp6 0 0 [::]:ssh [::]:* LISTEN 347/sshd: /usr/sbin
tcp6 0 0 [::]:33060 [::]:* LISTEN 6613/mysqld
udp 0 0 localhost:domain 0.0.0.0:* 178/systemd-resolve

and from juju's log:
2021-02-01 10:38:20 INFO juju-log Coordinator: Initializing coordinator layer
2021-02-01 10:38:20 INFO juju-log Invoking reactive handler: reactive/mysql_innodb_cluster_handlers.py:267:db_router_respond
2021-02-01 10:38:55 ERROR juju-log Cluster is unavailable: Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory
Traceback (most recent call last):
  File "<string>", line 2, in <module>
SystemError: RuntimeError: Dba.get_cluster: This function is not available through a session to a standalone instance (metadata exists, instance belongs to that metadata, but GR is not active)
2021-02-01 10:38:55 WARNING juju-log Cannot determine the cluster primary RW node for writes.