manila.share.manager logs "access denied" when using Ceph

[Giulio Fidente]

manila-share service fails initialization with:

2018-05-02 19:52:48.288 45 DEBUG manila.share.manager [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Start initialization of driver: 'CephFSDriver@hostgroup@cephfs' _driver_setup /usr/lib/python2.7/site-packages/manila/share/manager.py:303
2018-05-02 19:52:48.290 45 INFO manila.share.drivers.cephfs.driver [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] [cephfs}] Ceph client found, connecting...
2018-05-02 19:52:48.290 45 DEBUG ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Connecting to RADOS with config /etc/ceph/ceph.conf... connect /usr/lib/python2.7/site-packages/ceph_volume_client.py:449
2018-05-02 19:52:48.348 45 DEBUG ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Connection to RADOS complete connect /usr/lib/python2.7/site-packages/ceph_volume_client.py:458
2018-05-02 19:52:48.348 45 DEBUG ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Connecting to cephfs... connect /usr/lib/python2.7/site-packages/ceph_volume_client.py:460
2018-05-02 19:52:48.348 45 DEBUG ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] CephFS initializing... connect /usr/lib/python2.7/site-packages/ceph_volume_client.py:462
2018-05-02 19:52:48.350 45 DEBUG ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Premount eviction of manila starting connect /usr/lib/python2.7/site-packages/ceph_volume_client.py:465
2018-05-02 19:52:48.351 45 INFO ceph_volume_client [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] evict clients with auth_name=manila
2018-05-02 19:52:48.354 45 ERROR manila.share.manager [req-bc4bfb71-c3a6-498c-8d57-097b5dbc32b6 - - - - -] Error encountered during initialization of driver CephFSDriver@hostgroup@cephfs: Error: access denied
2018-05-02 19:52:48.354 45 ERROR manila.share.manager Traceback (most recent call last):
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 305, in _driver_setup
2018-05-02 19:52:48.354 45 ERROR manila.share.manager self.driver.do_setup(ctxt)
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/driver.py", line 125, in do_setup
2018-05-02 19:52:48.354 45 ERROR manila.share.manager ceph_vol_client=self.volume_client)
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/driver.py", line 195, in volume_client
2018-05-02 19:52:48.354 45 ERROR manila.share.manager self._volume_client.connect(premount_evict=premount_evict)
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 466, in connect
2018-05-02 19:52:48.354 45 ERROR manila.share.manager self.evict(premount_evict)
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 391, in evict
2018-05-02 19:52:48.354 45 ERROR manila.share.manager mds_map = self._rados_command("mds dump", {})
2018-05-02 19:52:48.354 45 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1277, in _rados_command
2018-05-02 19:52:48.354 45 ERROR manila.share.manager raise rados.Error(outs)
2018-05-02 19:52:48.354 45 ERROR manila.share.manager Error: access denied
2018-05-02 19:52:48.354 45 ERROR manila.share.manager

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/566462

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/566717

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566462
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=1dedf30fddc61b2f9793fb431bc0044c2d7c17ae
Submitter: Zuul
Branch: master

commit 1dedf30fddc61b2f9793fb431bc0044c2d7c17ae
Author: Giulio Fidente <email address hidden>
Date: Sun May 6 11:14:52 2018 +0200

    Do not overescape the Manila keyring caps

    Recent changes [1] [2] in ceph-ansible break old way of escaping
    quotes in the Manila keyring caps.

    1. https://github.com/ceph/ceph-ansible/commit/82ccbdafbcbba788cc46b8ef4166600ff80b5729
    2. https://github.com/ceph/ceph-ansible/commit/424815501a0c6072234a8e1311a0fefeb5bcc222

    Change-Id: I7ff5df0e602aff000c9a4213231b7abc68871fc4
    Closes-Bug: 1769436

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566717
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=130e65ff8db01bfce6b600a2bd1a3f1a769752e3
Submitter: Zuul
Branch: stable/queens

commit 130e65ff8db01bfce6b600a2bd1a3f1a769752e3
Author: Giulio Fidente <email address hidden>
Date: Sun May 6 11:14:52 2018 +0200

    Do not overescape the Manila keyring caps

    Recent changes [1] [2] in ceph-ansible break old way of escaping
    quotes in the Manila keyring caps.

    1. https://github.com/ceph/ceph-ansible/commit/82ccbdafbcbba788cc46b8ef4166600ff80b5729
    2. https://github.com/ceph/ceph-ansible/commit/424815501a0c6072234a8e1311a0fefeb5bcc222

    Change-Id: I7ff5df0e602aff000c9a4213231b7abc68871fc4
    Closes-Bug: 1769436
    (cherry picked from commit 1dedf30fddc61b2f9793fb431bc0044c2d7c17ae)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b3 development milestone.