On Folsom, I was having trouble working with Cinder. Volume creation would show "error" state, existing volumes could not be attached to instances, and attempting to delete existing volumes would show them hung in a "deleting" state. I tracked the problem back to the cinder-volume service, which was failing on startup, yielding the following stack trace:
CRITICAL cinder [-] Got unknown keyword args to utils.execute: {'old_name': None}
TRACE cinder Traceback (most recent call last):
TRACE cinder File "/usr/bin/cinder-volume", line 48, in <module>
TRACE cinder service.wait()
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 422, in wait
TRACE cinder _launcher.wait()
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 127, in wait
TRACE cinder service.wait()
TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait
TRACE cinder return self._exit_event.wait()
TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
TRACE cinder return hubs.get_hub().switch()
TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch
TRACE cinder return self.greenlet.switch()
TRACE cinder File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main
TRACE cinder result = function(*args, **kwargs)
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 88, in run_server
TRACE cinder server.start()
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 159, in start
TRACE cinder self.manager.init_host()
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 101, in init_host
TRACE cinder self.driver.ensure_export(ctxt, volume)
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 352, in ensure_export
TRACE cinder old_name=old_name )
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/iscsi.py", line 198, in create_iscsi_target
TRACE cinder self._new_target(name, tid, **kwargs)
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/iscsi.py", line 211, in _new_target
TRACE cinder **kwargs)
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/volume/iscsi.py", line 60, in _run
TRACE cinder self._execute(self._cmd, *args, run_as_root=True, **kwargs)
TRACE cinder File "/usr/lib/python2.7/dist-packages/cinder/utils.py", line 146, in execute
TRACE cinder 'to utils.execute: %r') % kwargs)
TRACE cinder Error: Got unknown keyword args to utils.execute: {'old_name': None}
It seems that the fix for https://bugs.launchpad.net/cinder/+bug/1065702 might have caused a side effect that led to the service failing to start.
In cinder/volume/driver.py, the "ISCSIDriver.ensure_export" method passes an "old_name" parameter to the "tgtadm.create_iscsi_target(...)" method. This method does not handle the parameter explicitly, but accepts it as a **kwargs dictionary which is neither accessed nor handled, but merely passed along to driver.py.
So, this "old_name" : None mapping improperly propagates down the call stack until it reaches utils.py, where it triggers an error as an unknown keyword and causes the service to fail.
Since the method where the bug originates seems to indicate the value is not even needed, and since it is never accessed or handled down the call stack, I was able to remove it from the method with no apparent negative consequences, like so:
# NOTE(jdg): For TgtAdm case iscsi_name is the ONLY param we need
# should clean this all up at some point in the future
self.tgtadm.create_iscsi_target(iscsi_name, iscsi_target,
0, volume_path,
check_exit_code=False,
old_name=old_name # REMOVE THIS LINE
)
With this parameter removed, I was able to successfully start my cinder-volume service to start normally and immediately begin to resolve its backlog of volume creation and deletion tasks.
Hi Alexander,
Yes, I came across this via a post on the mail-list yesterday but hadn't had time to file a bug yet. Thanks for logging this.