Add support for erasure-coded pool backend

[Przemyslaw Hausman]

This is a feature request for support erasure-coded pools.

Erasure-coded pools require less storage space compared to replicated pools. Currently, cinder-ceph supports only replicated pools.

The goal is to set up erasure-coded pool as a backend for Cinder. The problem is that images can only be created in a replicated pool due to lack of omap support in a an erasure coded pool. To overcome this issue, one must create two pools:

1. Replicated metadata pool, e.g 'cinder-ceph-ec-metadata',
2. Erasure-coded data pool,e.g 'cinder-ceph-ec-data'.

With such setup, the following configuration should be rendered by the charm:

/etc/cinder/cinder.conf:

[cinder-ceph-ec-metadata]
volume_backend_name = cinder-ceph-ec-metadata
volume_driver = cinder.volume.drivers.rbd.RBDDriver
rbd_pool = cinder-ceph-ec-metadata
rbd_user = cinder-ceph-ec-metadata
rbd_secret_uuid = <uuid>
rbd_ceph_conf = /var/lib/charm/cinder-ceph-ec-metadata/ceph.conf
[...]

/var/lib/charm/cinder-ceph-ec-metadata/ceph.conf:

[client.cinder-ceph-ec-metadata]
rbd default data pool = cinder-ceph-ec-data

Currently, rendering configuration to ceph.conf is not implemented.

For the initial implementation, cinder-ceph could support already existing, erasure-coded pools. As a next step, cinder-ceph could support creating metadata and data pools.

[Márton Kiss (marton-kiss)]

I would rather break up the request into two separate steps:
1, implement the missing "rbd default data pool" option into cinder-ceph's ceph conf, because at least it allows to use the erasure coding for pre-created data and metadata pools (external ceph clusters, or pools created as a post configuration step)
- add the rbd-default-data-pool variable to config.yaml
- render the following conditional part into ceph.conf:

[client]
rbd default data pool = {{ rbd_default_data_pool }}

In this case, the [client.{{poolname}}] entry is obsolote, because the cinder-ceph app defines a single ceph pool only. Multiple pools can be used by multiple cinder-ceph entries with different configurations. A properly configured cinder-ceph can result the following EC pool volume:

$ rbd ls cinder-ceph-ec-meta
volume-d6d12723-46f0-4e40-8941-90a754b59f51
$ rbd info cinder-ceph-ec-meta/volume-d6d12723-46f0-4e40-8941-90a754b59f51
rbd image 'volume-d6d12723-46f0-4e40-8941-90a754b59f51':
        size 1GiB in 256 objects
        order 22 (4MiB objects)
        data_pool: cinder-ceph-ec-data <--- the data pool name is here
        block_name_prefix: rbd_data.16.16da6b8b4567
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, data-pool <--- the data pool flag is there
        flags:
        create_timestamp: Thu Feb 20 17:35:20 2020

2, implement the proper EC pool creation for the cinder ceph charm. Basics seems to be already there for ceph-proxy, cinder-ceph, however for pool creation it calls the deprecated add_op_create_pool function:
https://github.com/openstack/charm-cinder-ceph/blob/master/hooks/cinder_hooks.py#L112

The add_op_create_pool *always* creates a replicated pool:
https://github.com/openstack/charm-cinder-ceph/blob/master/charmhelpers/contrib/storage/linux/ceph.py#L1219

So for the proper EC feature, additional config variables must be added to the cinder-ceph charm code, and properly invoke either the add_op_create_replicated_pool() or the add_op_create_erasure_pool() calls.

[Ryan Beisner (1chb1n)]

I think this is a large enough piece of work to warrant a design and specification conversation, per:
https://docs.openstack.org/charm-guide/latest/feature-specification.html

[James Page (james-page)]

Agree with @beisner - lets have a proper spec for this work so the design can be discussed and documented outside of a bug report.

[James Page (james-page)]

based on <conversations> the short term workaround solution should be via a 'config-flags' style configuration option in this charm - use of this for the EC pool configuration will be superceeded by the full EC support at a future date.

[Launchpad Janitor (janitor)]

[Expired for OpenStack cinder-ceph charm because there has been no activity for 60 days.]