Cinder Ceph backend doesn't provide not regular keyring files

[daniel.pawlik]

Cinder Ceph backend driver doesn't provide a path for keyring file.

Example:
/etc/cinder/cinder.conf :

[DEFAULT]
backup_ceph_user=cinder
backup_ceph_conf=/etc/ceph/ceph.test1234.conf
backup_driver=cinder.backup.drivers.ceph
backup_ceph_pool=volumes_backup
backup_ceph_chunk_size=134217728
backup_ceph_stripe_unit=0
backup_ceph_stripe_count=0
...

The ceph conf file is not regular, so Cinder raise an error:
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd [req-68d2ab9f-f7ab-47f4-a10c-9890694b37cf 0a5af0dc0ba64a6eb345d5e2844f22ad 9eec40c86a92477491ad3ee35b941be8 - e23bb165352648d088c2a928dcbb607c e23bb165352648d088c2a928dcbb607c] Error connecting to ceph cluster.
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd Traceback (most recent call last):
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd File "/usr/lib/python2.7/dist-packages/os_brick/initiator/linuxrbd.py", line 80, in connect
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd client.connect()
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd File "rados.pyx", line 785, in rados.Rados.connect (/build/ceph-10.2.3/src/build/rados.c:10073)
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd Error: error connecting to the cluster: error code 95
2017-02-27 07:48:57.163 25631 ERROR os_brick.initiator.linuxrbd
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server [req-68d2ab9f-f7ab-47f4-a10c-9890694b37cf 0a5af0dc0ba64a6eb345d5e2844f22ad 9eec40c86a92477491ad3ee35b941be8 - e23bb165352648d088c2a928dcbb607c e23bb165352648d088c2a928dcbb607c] Exception during message handling
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 404, in create_backup
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server self._update_backup_error(backup, six.text_type(err))
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 398, in create_backup
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server self._run_backup(context, backup, volume)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 434, in _run_backup
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server properties, is_snapshot)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 904, in _attach_device
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server return self._attach_volume(context, backup_device, properties)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 931, in _attach_volume
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server {'volume_id', volume.id})
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server self.force_reraise()
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 920, in _attach_volume
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server return self._connect_device(conn)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 943, in _connect_device
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server vol_handle = connector.connect_volume(conn['data'])
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/os_brick/utils.py", line 137, in trace_logging_wrapper
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/os_brick/initiator/connectors/rbd.py", line 148, in connect_volume
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server rbd_handle = self._get_rbd_handle(connection_properties)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/os_brick/initiator/connectors/rbd.py", line 95, in _get_rbd_handle
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server rbd_cluster_name=str(cluster_name))
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/os_brick/initiator/linuxrbd.py", line 60, in __init__
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server self.client, self.ioctx = self.connect()
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/os_brick/initiator/linuxrbd.py", line 88, in connect
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server raise exception.BrickException(message=msg)
2017-02-27 07:48:57.396 25631 ERROR oslo_messaging.rpc.server BrickException: Error connecting to ceph cluster.

It is because os-brick library doesn't follow the path but is searching for a keyring file in /etc/ceph/<clustername>.client.<user>.keyring file.

[Jon Bernard (jbernard)]

https://review.openstack.org/#/c/447077/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/447077
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=7b9a6686bcb5ead98934b737d1523601dea33d16
Submitter: Jenkins
Branch: master

commit 7b9a6686bcb5ead98934b737d1523601dea33d16
Author: Jon Bernard <email address hidden>
Date: Fri Mar 17 13:25:03 2017 -0400

    RBD: consider a custom keyring in connection info

    If a 'keyring' key is found in the connection info passed to
    connect_volume() use its value as the path to the keyring instead of the
    default location (/etc/ceph/<cluster>.client.<user>.keyring).

    This allows services such as cinder's RBD and Ceph backup drivers to
    make use of a custom keyring path that an admin has defined.

    Change-Id: Ib1230d3e40f56371567e1aead40db59667bad295
    Closes-bug: #1668304

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/451201

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/456672

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-brick 1.12.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/465044

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/465044
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=13f27658634beb2b33412c5d2ffb51f473212da2
Submitter: Jenkins
Branch: master

commit 13f27658634beb2b33412c5d2ffb51f473212da2
Author: Daniel Pawlik <email address hidden>
Date: Tue May 16 13:29:04 2017 +0200

    Changed way of providing RBD keyring from keyring_path to client token

    In created temporary file with RBD configuration there was provided
    a path for keyring file. It could work only when both system e.g.
    cinder-volume and host had exact same Ceph configuration, but there was no
    guarantee for that.
    Providing Ceph token will help avoid such situations.

    Partial-Bug: #1668304
    Change-Id: I465828dec58ab2110b33743a10e6fc518b5c85ff

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/451201
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=51a6234d7863f20c76e809dfa3cf7439a36c645d
Submitter: Jenkins
Branch: stable/ocata

commit 51a6234d7863f20c76e809dfa3cf7439a36c645d
Author: Jon Bernard <email address hidden>
Date: Fri Mar 17 13:25:03 2017 -0400

    RBD: consider a custom keyring in connection info

    If a 'keyring' key is found in the connection info passed to
    connect_volume() use its value as the path to the keyring instead of the
    default location (/etc/ceph/<cluster>.client.<user>.keyring).

    This allows services such as cinder's RBD and Ceph backup drivers to
    make use of a custom keyring path that an admin has defined.

    Change-Id: Ib1230d3e40f56371567e1aead40db59667bad295
    Closes-bug: #1668304
    (cherry picked from commit 7b9a6686bcb5ead98934b737d1523601dea33d16)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/456672
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=d0520a07e9dcee53fe2f13900f4c36c7e455c6f0
Submitter: Jenkins
Branch: master

commit d0520a07e9dcee53fe2f13900f4c36c7e455c6f0
Author: Daniel Pawlik <email address hidden>
Date: Thu Apr 13 15:32:05 2017 +0000

    Add custom keyring when initializing RBD connection

    Added RBD keyring configuration parameter which will help administrator
    set an custom keyring path to the Ceph cluster.

    Closes-bug: #1668304

    Change-Id: I263cc10dc877b20bbc205ea55173ad3878687ea1

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 11.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-brick 1.11.1 release.