ceph-osd: support use of directly backed OSD's on bcache devices

[JuanJo Ciarlante]

If you specify an non-existent directory to ceph-osd as e.g.:
osd-devices="... /path/to/foo", the charm will create /path/to/foo
and start using it - this behavior is different from the device
case (which will obviously skip if not present), and should match it.

Use case: because current ceph-osd charm doesn't support actions
to add/rm osds, to add new devices to the pool we set osd-devices
to include the new devices added to *some* nodes, then config-changed
will add the present ones -- but then, if it's a directory, it'll
behave as above, creating it if not present, and start using it.

In our specific case, these were bcache backed directories, and
because of http://tracker.ceph.com/issues/13278 we needed to have
them properly mounted to be able to add osds to them - then we
ended filing up the rootFS on the nodes where these mounted dirs
had not been setup yet.

[Chris Holcombe (xfactor973)]

I'd be happy to give you a hand with creating a PR to add add/rm osd actions if you're interested.

[Chris Holcombe (xfactor973)]

Usually maas handles taking care of bcached backed directories for the charm. In your case though it sounds like you don't have that. I'll have to put some thought into how we can handle that use case. Ideas are welcome :)

[James Page (james-page)]

The directly backed OSD is a testing feature only; I'm pretty happy with the current function that creates the directory for the OSD if its not found.

This feels like a boot ordering type issue with directly backed OSD's on non-root disks, but I'm not 100% sure how we fix that effectively.

[James Page (james-page)]

I guess the question really is whether we want to support use of bcache devices for OSD's - the alternative is to not use the whole of the underlying SSD for bcache, reserving part of it for use by ceph as a journal device for underlying SATA or SAS spindles.

[Chris MacNaughton (chris.macnaughton)]

Marking the charm-ceph task wontfix as the ceph charm has been removed from support for a while now

[Chris MacNaughton (chris.macnaughton)]

Rather than adding a ceph-osd bug target, I'm leaving this one as is as we do fully support bcache backed devices now