bootstrap-source relation should provide sanity check for compatible ceph version

[Drew Freiberger]

In the field as I was migrating from charm-ceph to charm-ceph-mon on a trusty-mitaka cloud, I deployed cs:trusty/ceph-mon, then juju add-relation ceph-mon ceph (which auto-relates on bootstrap-source interface).

On the ceph application, the source was set to a UCA link cloud:trusty-updates which installed ceph 10.2.11, but on ceph-mon, the source was left default (operator error) which installed 0.80.11 on the new trusty unit deployed.

After expanding ceph-mon to 3 units with juju add-unit -n 2 ceph-mon, we found that the 3 ceph-mon nodes had clustered and show 4 mons with 1 mon down and 0 OSDs, but the ceph/0 unit still had only 1 mon defined in ceph status and in /etc/ceph/ceph.conf even after properly being triggered in the relation.

I believe it would be beneficial to the product user base to alert to operator error by providing a sanity check for version of ceph-mon on either side of the bootstrap-source interface before succeeding on bootstrap-source-relation-joined. Perhaps setting a blocked state waiting for version matching on the "older" (pkgrev-wise) cluster.

[Drew Freiberger (afreiberger)]

To reproduce:

juju deploy cs:trusty/ceph --config monitor-count=1 --config source='cloud:trusty-updates'
juju deploy ceph-osd
juju add-relation ceph-osd ceph
juju deploy cs:trusty/ceph-mon --config no-bootstrap=true
juju add-unit ceph-mon -n 2

This should present the error state. This is from the top of my head and not proven to repro.

[Ryan Beisner (1chb1n)]

Agree, we should guard around this in the charm. Thank you for the bug.

[Ryan Beisner (1chb1n)]

FYI charm-ceph is deprecated, and will not receive further patches (charm-ceph-mon and charm-ceph-osd are supported, and should be the focus for this).