Comment 1 for bug 2009968

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

I think this is a dup of "During scale-out of cluster (zaza-openstack-tests) the leader fails to join in the new instance when related to prometheus" (https://bugs.launchpad.net/charm-mysql-innodb-cluster/+bug/2015256) which is a race between the prometheus hook and joining the instance into the cluster from the leader (which would be on different units for the bug to happen).

The first test that failed is the add unit test, which is when it all goes to pot.

2023-03-09 15:21:55.490440 | focal-medium | 2023-03-09 15:21:55 [INFO] test_801_add_unit (zaza.openstack.charm_tests.mysql.tests.MySQLInnoDBClusterScaleTest)
2023-03-09 15:21:55.490501 | focal-medium | 2023-03-09 15:21:55 [INFO] Add mysql-innodb-cluster node.
2023-03-09 15:21:55.490520 | focal-medium | 2023-03-09 15:21:55 [INFO] ...
2023-03-09 15:21:55.490534 | focal-medium | 2023-03-09 15:21:55 [INFO] Wait till model is idle ...
2023-03-09 15:21:55.993575 | focal-medium | 2023-03-09 15:21:55 [INFO] Adding unit after removed unit ...
2023-03-09 15:21:56.325775 | focal-medium | 2023-03-09 15:21:56 [INFO] Wait until 3 units ...
2023-03-09 15:21:56.416060 | focal-medium | 2023-03-09 15:21:56 [INFO] Wait for application states ...
2023-03-09 15:21:56.416955 | focal-medium | 2023-03-09 15:21:56 [INFO] Waiting for application states to reach targeted states.
2023-03-09 15:21:56.421116 | focal-medium | 2023-03-09 15:21:56 [INFO] Waiting for an application to be present
2023-03-09 15:21:56.421804 | focal-medium | 2023-03-09 15:21:56 [INFO] Now checking workload status and status messages
2023-03-09 15:21:56.930799 | focal-medium | 2023-03-09 15:21:56 [INFO] Application prometheus2 is ready.
2023-03-09 15:21:56.936002 | focal-medium | 2023-03-09 15:21:56 [INFO] Application keystone is ready.
2023-03-09 15:21:56.939789 | focal-medium | 2023-03-09 15:21:56 [INFO] Application keystone-mysql-router is ready.
2023-03-09 15:21:56.953975 | focal-medium | 2023-03-09 15:21:56 [INFO] Application vault is ready.
2023-03-09 15:21:56.957646 | focal-medium | 2023-03-09 15:21:56 [INFO] Application vault-mysql-router is ready.

Marking as a duplicate for the moment; but please un-dup if you feel it is different.