bootstrapped-osds in ceph-mon relation does not equal number of OSDs in status
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceph OSD Charm |
New
|
Undecided
|
Unassigned |
Bug Description
ceph-mon/0 waiting executing 0/lxd/0 1.2.3.4 Monitor bootstrapped but waiting for number of OSDs to reach expected-osd-count (38)
I'm seeing the above error when upgrading charms from 2018 versions to 21.01.
It seems that there is a discrepancy or non-updating relation when additional OSDs are identified.
What I see is that my mon shows it's expecting 38 OSDS, and when I count all of the 'Unit is ready (3 OSD)' and 'Unit is ready (2 OSD)' statuses of all of my related ceph-osd units, they add up to 38, however, when I inspect the relation information, I see that there are only "2" bootstrapped-osds showing in each unit's relation.
I've worked around this by running a relation-set on the units with 3 OSDS to update bootstrapped-
This has mostly (maybe only?) been seen in environments upgrading to a newer ceph-mon that supports waiting for expected OSD count before bootstrapping from a version that was prior to that functionality.