Comment 10 for bug 1828534

Reproducing with designate, the problem appears after we have one designate unit that is at bionic-rocky (py3) and one that is at bionic-queens (py2). It is resolved once all units are at bionic-rocky. More details on that:

# upgrade one unit to bionic-rocky
juju config designate openstack-origin=cloud:bionic-rocky
juju run-action designate/0 openstack-upgrade

juju status
Unit Workload Agent Machine Public address Ports Message
designate/0 error idle 7 10.5.0.198 9001/tcp Services not running that should be: designate-producer
  hacluster-designate/0 active idle 10.5.0.198 Unit is ready and clustered
designate/1* active idle 8 10.5.0.6 9001/tcp Unit is ready
  hacluster-designate/1* active idle 10.5.0.6 Unit is ready and clustered

juju ssh designate/0

2019-07-25 18:04:27.452 24807 DEBUG designate.coordination [-] CCB: list(self._get_members(self._group_id))=[b'juju-3d28eb-coreycb2-8:86c49114-a37a-4e1a-8654-72bf1ccbde1f', 'juju-3d28eb-coreycb2-7:d64cde68-d9f0-4910-9f1a-6d9724ea77b9'] _update_partitions /usr/lib/python3/dist-packages/designate/coordination.py:202

where:
- machine 8 (b'juju-3d28eb-coreycb2-8:86c49114-a37a-4e1a-8654-72bf1ccbde1f') hasn't been upgraded yet and is still bionic-queens
- machine 7 ('juju-3d28eb-coreycb2-7:d64cde68-d9f0-4910-9f1a-6d9724ea77b9') is the machine that was upgraded to bionic-rocky

Note: designate coordinator backend gets the juju strings by calling get_members() from the tooz memcached backend. /usr/lib/python3/dist-packages/tooz/drivers/memcached.py

# now try to upgrade the other unit to bionic-rocky
juju run-action designate/1 openstack-upgrade

# restart memcache and designate-producer on designate/0 and resolve
juju resolved designate/0

# all better

Unit Workload Agent Machine Public address Ports Message
designate/0 active idle 7 10.5.0.198 9001/tcp Unit is ready
  hacluster-designate/0 active idle 10.5.0.198 Unit is ready and clustered
designate/1* active idle 8 10.5.0.6 9001/tcp Unit is ready
  hacluster-designate/1* active idle 10.5.0.6 Unit is ready and clustered

juju ssh designate/0

2019-07-25 19:31:11.365 12217 DEBUG designate.coordination [-] CCB: list(self._get_members(self._group_id))=['juju-3d28eb-coreycb2-7:0ceca5d8-9eee-492d-bead-f99ee990a21b', 'juju-3d28eb-coreycb2-8:134e6142-4364-43a5-96fc-b9f125ba87a0'] _update_partitions /usr/lib/python3/dist-packages/designate/coordination.py:202

Both strings are str type now and able to be sorted.