Conflicting Transaction Sets following Complete Outage of InnoDB Cluster

[David Ames]

[Impact]
The 8.0.18 version of mysql-8.0 has this upstream bug:

https://bugs.mysql.com/bug.php?id=97279

Note 8.0.17 did not display this bug.

After a complete outage dba.rebootClusterFromCompleteOutage() errors with:

output: "ERROR: Conflicting transaction sets between 10.5.0.55:3306 and 10.5.0.52:3306 Dba.rebootClusterFromCompleteOutage: Conflicting transaction sets between 10.5.0.55:3306 and 10.5.0.52:3306 (MYSQLSH 51152) at /tmp/tmpn2y5qo9l.js:3:12 in dba.rebootClusterFromCompleteOutage();

[Regression Potential]
TBD

[Fix]

[Test Case]
* Build an InnoDB cluster
  - var cluster = dba.createCluster('cluster')
  - cluster.addInstance('<email address hidden>:xxxx')
  - cluster.addInstance('<email address hidden>:yyyy')
  - cluster.setOption("consistency", "BEFORE")
* Shut each node of the cluster down (reboot)
* Start mysql on each node
* (Maybe?) INSERT some random data into the same table on all cluster instances
* Run dba.rebootClusterFromCompleteOutage()

The cs:~openstack-charmers-next/mysql-innodb-cluster has a test zaza.openstack.charm_tests.mysql.tests.MySQLInnoDBClusterColdStartTest that re-creates the failure. Note: It is currently disabled due to this bug.

[Workaround]
From upstream bug report,
"""
  a) Shutdown the instance that contains the GTID subset (typically a secondary that was behind, or a primary that was ejected from the cluster)
  b) Execute rebootClusterFromCompleteOutage to bring the cluster up
  c) Start the instance from step a)
  d) Execute a cluster.rescan() to bring the instance back into the cluster
"""

[Bryce Harrington (bryce)]

A patch is not yet available for this issue from upstream, however it is confirmed by them with their own test case defined, and a workaround, both of which I've updated in the bug description. I've linked the upstream bug to hopefully give visibility when a patch does become available.

[Alex Lutay (taurus)]

The issue can be resolved, as upstream fixed it in 8.0.19 and the current MySQL version is 8.0.33 in Ubuntu[security].