Access denied for clustering when unit has multiple nics

[Xav Paice]

While deploying a 3 unit cluster via openstack-on-lxd adapted bundle, the logs on one of the 3 units showed:

2020-09-15 07:52:12 ERROR juju-log Cluster is unavailable: Logger: Tried to log to an uninitialized logger.
ESC[33mWARNING: ESC[0mCluster error connecting to target: MySQL Error 1045 (28000): Access denied for user 'clusteruser'@'10.0.8.81' (using password: YES)
Traceback (most recent call last):
  File "<string>", line 2, in <module>
mysqlsh.DBError: MySQL Error (1045): Dba.get_cluster: Access denied for user 'clusteruser'@'10.0.8.81' (using password: YES)

The cluster was, as a result, unable to form.

The openstack-on-lxd lxd-profile.yaml has two NICs connected to the same network - the unit had two IP addresses, same subnet, and the access was configured for the address on eth0 (10.0.8.240/24), but not on eth1 (10.0.8.81). I stopped eth1, and the cluster completed.

Given the somewhat unique situation of having two NICs on the same network (to allow for neutron gateway to have an eth1 for physnet1), I'm not sure this is even a situation I'd expect to see in a real world scenario. However, any cloud that doesn't support spaces, but does have multiple nics, will see this.

[Xav Paice (xavpaice)]

While attempting to reproduce, I was unable to collect the original error message. Here's the setup I used, and note that the cluster still doesn't form even though the message reported earlier isn't in the logs.

juju deploy --series focal cs:ubuntu -n 3
juju run --application ubuntu 'echo "network: {ethernets: {eth1: {dhcp4: true}}}" |sudo tee /etc/netplan/eth1.yaml'
juju run --application ubuntu 'sudo netplan apply'
juju deploy -n 3 --to 0,1,2 cs:mysql-innodb-cluster

Model Controller Cloud/Region Version SLA Timestamp
default charmlab-lxd charmlab/default 2.9-beta1 unsupported 16:20:13+13:00

App Version Status Scale Charm Store Rev OS Message
mysql-innodb-cluster 8.0.21 waiting 3 mysql-innodb-cluster jujucharms 1 ubuntu Not all instances configured for clustering
ubuntu 20.04 active 3 ubuntu jujucharms 15 ubuntu ready

Unit Workload Agent Machine Public address Ports Message
mysql-innodb-cluster/0* waiting idle 3 10.0.8.195 Not all instances configured for clustering
mysql-innodb-cluster/1 waiting idle 4 10.0.8.245 Instance not yet configured for clustering
mysql-innodb-cluster/2 waiting idle 5 10.0.8.58 Instance not yet configured for clustering
ubuntu/3* active idle 3 10.0.8.195 ready
ubuntu/4 active idle 4 10.0.8.245 ready
ubuntu/5 active idle 5 10.0.8.58 ready

Will attach juju-crashdump.

[Alex Kavanagh (ajkavanagh)]

I don't think we're going to fix (or add a feature to address ) this issue, as it is a pretty unique situation. However, if this becomes a more significant issue, then please re-open the bug.