cinder-volume fails to start in Ocata when no backend is connected

[Liam Young]

When deploying the cinder charm using Ocata and no backend charm connected, the cinder-volume service fails to start. Error from cinder-volume.log:

Configuration for cinder-volume does not specify "enabled_backends". Using DEFAULT section to configure drivers is not supported since Ocata.
No volume service(s) started successfully, terminating.

The charm eventually reports the dead services via juju status:

Services not running that should be: cinder-volume

The easiest way to see this is to run the amulet test tests/gate-basic-xenial-ocata. It fails with

2017-02-16 09:17:07,356 create_cinder_volume DEBUG: Creating cinder volume...
2017-02-16 09:17:14,114 resource_reaches_status DEBUG: Volume status wait status check: 0 [creating:available] 0974c72f-e88d-40c2-a00a-effa49abee9f
2017-02-16 09:17:18,304 resource_reaches_status DEBUG: Volume status wait status check: 1 [error:available] 0974c72f-e88d-40c2-a00a-effa49abee9f
....
2017-02-16 09:19:19,590 resource_reaches_status DEBUG: Volume status wait: expected, actual status = error, available
2017-02-16 09:19:19,590 resource_reaches_status DEBUG: 0974c72f-e88d-40c2-a00a-effa49abee9f never reached expected status: available

[Liam Young (gnuoy)]

I've looked into this and it seems more involved than I had first thought. The requirement to use enabled_backends causes volumes to be created with a different os-vol-host-attr:host. If a cloud is upgraded to Ocata any existing volumes will have an invalid os-vol-host-attr:host. The existing volumes need to have that attribute updated via cinder-manage *1

*1 https://docs.openstack.org/admin-guide/blockstorage-multi-backend.html#enable-multiple-storage-back-ends

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/435459
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=8641e81498dce5a7fa2b42bb48aa9655c5df7d04
Submitter: Jenkins
Branch: master

commit 8641e81498dce5a7fa2b42bb48aa9655c5df7d04
Author: Liam Young <email address hidden>
Date: Thu Feb 16 12:13:03 2017 +0000

    Move backends into their own sections

    Using DEFAULT section to configure drivers is not supported since
    Ocata. This change lists backends in enabled_backends.

    Note: Using sectional config causes the os-vol-host-attr:host
    volume attribute to change which invalidates existing volumes.
    A subsequant change is needed to add an action for renaming
    the host attribute of existing volumes *1

    *1 https://docs.openstack.org/admin-guide/blockstorage-multi-backend.html#enable-multiple-storage-back-ends

    Partial-Bug: #1665272
    Change-Id: I22c3e74b0874c051bee89e4609088facf95b4664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/436848

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/436848
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=d21160a0d354a815b563d55a33d204c210e815cf
Submitter: Jenkins
Branch: master

commit d21160a0d354a815b563d55a33d204c210e815cf
Author: Liam Young <email address hidden>
Date: Wed Feb 22 09:48:08 2017 +0000

    Add actions for renaming volume host attr

    Add two new actions which are essentially wrappers around

    cinder-manage volume update_host \
        --currenthost CURRENTHOST \
        --newhost NEWHOST

    In previous versions of the charm if block-device is set or the
    legacy ceph relation is used (as opposed to storage-backend relation)
    then the configuration of those backends is done in the [DEFAULT]
    section of the cinder.conf. As of Ocata that is no longer supported
    and backends need to be listed in their own sections and referenced
    via enable_backends parameter. This change in config results in a
    change of host name and existing volumes need to have their metadata
    updated to point at the new hostname.

    Old Hostname: <unit-name>
    New Hostname: <unit-name>@<backend section name>#<volume-backend-name>

    New Action: volume-backend-name
    Used for updating the host attribute of volumes to add the driver
    name. This is needed after an upgrade to Ocata if there are existing
    volumes which have been configured prior to multi-backends
    being enabled.

    New Action: rename-volume-host
    Used for updating the host attribute of volumes. This action is
    a lower level action then volume-host-add-driver and simply passes
    the old and new hosts verbatim to cinder-manage.

    Change-Id: I989074a3f41126aa57c514f7e18b887733bc18fe
    Partial-Bug: #1665272