Comment 0 for bug 2021880

When running actions such as remove-instance with force=true, the following flags are removed for the specified instance address:

cluster-instance-clustered-10-5-2-206: "True"
cluster-instance-configured-10-5-2-206: "True"

Then, if an action such as add-instance is run for same instance, the charm code does not recognize the updated flags, as can be seen in the logs:

2023-05-29 14:34:07 INFO unit.mysql/0.juju-log server.go:316 Configuring and adding instance to the cluster: 10.5.2.206.
2023-05-29 14:34:09 WARNING unit.mysql/0.juju-log server.go:316 User clusteruser exists.
2023-05-29 14:34:11 WARNING unit.mysql/0.juju-log server.go:316 User clusteruser exists.
2023-05-29 14:34:11 WARNING unit.mysql/0.juju-log server.go:316 Instance: 10.5.2.206, already configured.
2023-05-29 14:34:11 WARNING unit.mysql/0.juju-log server.go:316 Instance: 10.5.2.206, already clustered.

Even if the action is repeated multiple times, the result seen in the above logs is the same, despite the flags not existing anymore in leader-get.

Later, whenever an update-status hook runs (or is manually run using juju run -u mysql/leader -- ./hooks/update-status), the add-instance action recognizes the flags are not there anymore:

2023-05-29 14:34:55 INFO unit.mysql/0.juju-log server.go:316 Configuring and adding instance to the cluster: 10.5.2.206.
2023-05-29 14:34:57 WARNING unit.mysql/0.juju-log server.go:316 User clusteruser exists.
2023-05-29 14:34:59 WARNING unit.mysql/0.juju-log server.go:316 User clusteruser exists.
2023-05-29 14:35:00 INFO unit.mysql/0.juju-log server.go:316 Configuring instance for clustering: 10.5.2.206.

Therefore it is clear that the actions do not really set the flags as they are currently being read by the code using reactive.is_flag_set(). A hook such as update-status needs to run to set those flags.

I have tried hacking the code and inserting the method "update_dotted_flags" immediately at the end of remove-instance, but it still did not update the flags, as modifying the leader-settings is the same thing that both remove-instance and add-instance actions do. Apparently it is the fact that actions are not hook that do not get the flags saved, and a hook needs to run to read the updated leader-settings and apply them to flags.

[1] https://github.com/openstack/charm-mysql-innodb-cluster/blob/446eaabbfded2400245bbf630fb91418847f4c81/src/lib/charm/openstack/mysql_innodb_cluster.py#L2089