"rados lspools" in RBDDriver's check_for_setup_error() does not correctly set the Cephx client identity. Instead, it defaults to the "client.admin" identity. In a Ceph cluster where Cephx authentication is disabled, this does not matter -- client authentication succeeds no matter which client identity is set.
In a cluster where authentication is enabled, and the Cinder host does not have the client.admin's Cephx key available, "rados lspools" fails because the client.admin key is not available. This causes cinder-volume to fail on startup:
012-11-27 11:51:25 DEBUG cinder.utils [req-30c51712-e631-4034-98b3-686fb29705fd None None] Running cmd (subprocess): rados lspools execute /usr/lib/python2.7/dist-packages/cinder/utils.py:156
2012-11-27 11:51:25 DEBUG cinder.utils [req-30c51712-e631-4034-98b3-686fb29705fd None None] Result was 254 execute /usr/lib/python2.7/dist-packages/cinder/utils.py:172
2012-11-27 11:51:25 22750 CRITICAL cinder [-] Unexpected error while running command.
Command: rados lspools
Exit code: 254
Stdout: ''
Stderr: "2012-11-27 11:51:25.025430 7f18f3413780 -1 auth: failed to open keyring from /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin\n2012-11-27 11:51:25.025447 7f18f3413780 -1 monclient(hunting): failed to open keyring: (2) No such file or directory\n2012-11-27 11:51:25.025506 7f18f3413780 0 librados: client.admin initialization error (2) No such file or directory\ncouldn't connect to cluster! error -2\n"
2012-11-27 11:51:25 22750 TRACE cinder Traceback (most recent call last):
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/bin/cinder-volume", line 48, in <module>
2012-11-27 11:51:25 22750 TRACE cinder service.wait()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 422, in wait
2012-11-27 11:51:25 22750 TRACE cinder _launcher.wait()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 127, in wait
2012-11-27 11:51:25 22750 TRACE cinder service.wait()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait
2012-11-27 11:51:25 22750 TRACE cinder return self._exit_event.wait()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2012-11-27 11:51:25 22750 TRACE cinder return hubs.get_hub().switch()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch
2012-11-27 11:51:25 22750 TRACE cinder return self.greenlet.switch()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main
2012-11-27 11:51:25 22750 TRACE cinder result = function(*args, **kwargs)
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 88, in run_server
2012-11-27 11:51:25 22750 TRACE cinder server.start()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 159, in start
2012-11-27 11:51:25 22750 TRACE cinder self.manager.init_host()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 95, in init_host
2012-11-27 11:51:25 22750 TRACE cinder self.driver.check_for_setup_error()
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 614, in check_for_setup_error
2012-11-27 11:51:25 22750 TRACE cinder (stdout, stderr) = self._execute('rados', 'lspools')
2012-11-27 11:51:25 22750 TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/utils.py", line 179, in execute
2012-11-27 11:51:25 22750 TRACE cinder cmd=' '.join(cmd))
2012-11-27 11:51:25 22750 TRACE cinder ProcessExecutionError: Unexpected error while running command.
A workaround is to make the client.admin key available on the Cinder host, and make it readable to the "cinder" group (or whichever gid the cinder-volume process runs under). But the proper fix is for the "rados lspools" invocation to set a proper "-n" flag.
Fix proposed to branch: master /review. openstack. org/16962
Review: https:/