Deploying BlueStore OSDs when osd-journal is set but bluestore-db is not should block

[Trent Lloyd]

== Problem ==

If the charm is deploying a BlueStore OSD and the osd-journal configuration option is SET but the bluestore-db configuration is NOT SET, then the charm should error to ensure the administrator really wants to configure a BlueStore OSD with no NVMe device in use.

Currently the charm will happily deploy a BlueStore OSD with no NVMe usage and the administrator is likely not to notice until much later when the cluster usage has reached levels to make this obvious. In most cases it is not noticed initially.

This is likely to happen either when expanding the cluster with new nodes, or replacing failed disks using the add-disk action.

== Justification ==

There are multiple reasons the current behavior is problematic

Earlier version of the ceph-osd charm would deploy FileStore OSDs even on Luminous which newer charms deploy by default as BlueStore. As a result many ceph-osd applications are deployed with osd-journal set but with bluestore-db unset.

It is also likely to happen in scenarios where the ceph-osd charm is upgraded (since the default behavior changed) and also when deployments have undergone release upgrades to convert from jewel/luminous to luminous+.

This has happened in one specific production deployment where a second ceph-osd application was deployed to an existing model (to allow for different osd-devices to be set) however the new charm had the latest charm version and used BlueStore, but the old charm version used FileStore even though it was the same Ceph Luminous version.

Given that Xenial was often Jewel (or earlier) and that Bionic is often Luminous (or newer) there is likely to be an increasing number of deployments suffering from this issue after a release and/or series upgrade.