Add support for Cinder multi-backend

[Pedro Guimarães]

Hi,

Cinder may have multiple backends, with multiple types of storage.
CK should have an automatic mechanism to expose it via config.

For Cinder with multiple Ceph pools:

First, each Ceph pool should mapped into a cinder-ceph charm.
That will connect to either Ceph-proxy or ceph-mon and request for a given pool.

Once the charm is available, that will show up a new service backend on cinder, just run:
openstack volume service list

Following the documentation from OpenStack, we can map a backend into a volume type with:
openstack volume type create --property volume_backend_name=cinder-ceph-second my-new-volume-type
Doc: https://docs.openstack.org/cinder/latest/admin/blockstorage-multi-backend.html

Using volume_backend_name as a property to do the mapping. That also allows to have a per-tenant volume type.

To recover the property, find it on /etc/cinder/cinder.conf, for example:

[cinder-ceph]
volume_backend_name = cinder-ceph
volume_driver = cinder.volume.drivers.rbd.RBDDriver
. . . .

Once the volume type has been created, we can enforce K8s to choose one volume type as per code on:
https://github.com/kubernetes/cloud-provider-openstack/blob/45620373c17643a1ee2e821cec2c72b2b9776d9f/pkg/csi/cinder/controllerserver.go#L63

That means, with the following storageclass:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-sc-cinderplugin
provisioner: csi-cinderplugin
parameters:
  type: "my-new-volume-type"

Kubernetes will always pick up that volume type, which maps into an specific backend on Ceph.

That should take the form of a list such as:
config:
  cinder-multi-backend:
    - volume_backend_name: cinder-ceph
      volume_type: type-name
    - volume_backend_name: cinder-ceph-second
      volume_type: type-name-second

And as a result, it will create/destroy storage classes as config changes.