successfully bootstrapped mon cluster claims it is not so if num_units no longer matches monitor-count

[Edward Hope-Morley]

Here I have a 1 unit mon cluster with monitor-count=1 that has successfully deployed and clustered with status "Unit is ready" etc. Then I set monitor-count=3 and it says "Insufficient peer units to bootstrap cluster (require 3)" which is not true.

https://pastebin.ubuntu.com/p/prTCbTMsBk/

The same if I have 3 units with monitor-count=3 and i remove 1 unit:

https://pastebin.ubuntu.com/p/mM6cFCDKmq/

[Edward Hope-Morley (hopem)]

also see https://bugs.launchpad.net/charm-ceph-mon/+bug/1732491

[David Ames (thedac)]

TRIAGE:

Rework workload status such that it knows if we have fully clustered at some point in the past. Or reword the message to include that possibility.