Detaching volume fails after rebuilding the vm when nova uses vmwareapi.VMwareVCDriver as compute driver and cinder uses vmware.vmdk.VMwareVcVmdkDriver as volume driver

Bug #1712458 reported by wind
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
New
Undecided
Unassigned

Bug Description

I use vmwareapi.VMwareVCDriver as compute driver and vmware.vmdk.VMwareVcVmdkDriver as volume driver, when I try to detach volume from vm which has been rebuilded, I fail.

The nova-compute log shows:
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] Traceback (most recent call last):
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] encryption=encryption)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self._detach_volume_vmdk(connection_info, instance)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self.detach_disk_from_vm(vm_ref, instance, device)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] vm_util.reconfigure_vm(self._session, vm_ref, vmdk_detach_config_spec)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] session._wait_for_task(reconfig_task)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return self.wait_for_task(task_ref)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return evt.wait()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return hubs.get_hub().switch()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return self.greenlet.switch()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self.f(*self.args, **self.kw)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] raise task_ex
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] InvalidPowerStateException: 无法在当前状况 (已打开电源) 下执行尝试的操作。
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d]
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher payload)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in decorated_function
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in decorated_function
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4812, in detach_volume
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher self._detach_volume(context, volume_id, instance)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4697, in _driver_detach_volume
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher return self._volumeops.detach_volume(connection_info, instance)
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 597, in detach_volume
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher self._detach_volume_vmdk(connection_info, instance)
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 558, in _detach_volume_vmdk
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher self.detach_disk_from_vm(vm_ref, instance, device)
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 120, in detach_disk_from_vm
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher vm_util.reconfigure_vm(self._session, vm_ref, vmdk_detach_config_spec)
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vm_util.py", line 1469, in reconfigure_vm
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher return self.wait_for_task(task_ref)
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher return self.greenlet.switch()
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 428, in _poll_task
2017-08-22 16:15:12.260 2213990 ERROR oslo_messaging.rpc.dispatcher
2017-08-22 16:15:50.338 2213990 INFO nova.compute.resource_tracker [req-70c179da-c3ae-4b77-a0ed-9c08b2a596cb - - - - -] Total usable vcpus: 32, total allocated vcpus: 0
2017-08-22 16:16:04.512 2213990 ERROR oslo_vmware.common.loopingcall File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
2017-08-22 16:16:04.512 2213990 ERROR oslo_vmware.common.loopingcall self.f(*self.args, **self.kw)
2017-08-22 16:16:04.512 2213990 ERROR oslo_vmware.common.loopingcall File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 428, in _poll_task
2017-08-22 16:16:04.512 2213990 ERROR oslo_vmware.common.loopingcall
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] Traceback (most recent call last):
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] encryption=encryption)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self._detach_volume_vmdk(connection_info, instance)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self.detach_disk_from_vm(vm_ref, instance, device)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] vm_util.reconfigure_vm(self._session, vm_ref, vmdk_detach_config_spec)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] session._wait_for_task(reconfig_task)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return self.wait_for_task(task_ref)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return evt.wait()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return hubs.get_hub().switch()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] return self.greenlet.switch()
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] self.f(*self.args, **self.kw)
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] raise task_ex
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d] InvalidPowerStateException: 无法在当前状况 (已打开电源) 下执行尝试的操作。
2017-08-22 16:16:04.514 2213990 ERROR nova.compute.manager [instance: c3e4183c-bed8-4c45-b71e-83811456ca5d]
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher payload)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in decorated_function
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in decorated_function
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4812, in detach_volume
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher self._detach_volume(context, volume_id, instance)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4697, in _driver_detach_volume
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher encryption=encryption)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 416, in detach_volume
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return self._volumeops.detach_volume(connection_info, instance)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 597, in detach_volume
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher self._detach_volume_vmdk(connection_info, instance)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 558, in _detach_volume_vmdk
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher self.detach_disk_from_vm(vm_ref, instance, device)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/volumeops.py", line 120, in detach_disk_from_vm
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher vm_util.reconfigure_vm(self._session, vm_ref, vmdk_detach_config_spec)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vm_util.py", line 1469, in reconfigure_vm
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher session._wait_for_task(reconfig_task)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 641, in _wait_for_task
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return self.wait_for_task(task_ref)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 386, in wait_for_task
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return evt.wait()
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return hubs.get_hub().switch()
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher return self.greenlet.switch()
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher self.f(*self.args, **self.kw)
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 428, in _poll_task
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher raise task_ex
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher InvalidPowerStateException: 无法在当前状况 (已打开电源) 下执行尝试的操作。
2017-08-22 16:16:04.820 2213990 ERROR oslo_messaging.rpc.dispatcher

The steps to reproduce the bug:
1)create vm1 by "nova boot" command line
2)create volume1 by "cinder create" command line
3)attach volume1 to vm1 by "nova volume-attach"
4)rebuild vm1 by "nova rebuild"
5)detach volume1 from vm1 by "nova volume-detach"

I realize the vm is active ,so I shut down the vm ,then detaching volume successes.

The branch of OpenStack that I use is "liberty"

Revision history for this message
Hanxi Liu (hanxi-liu) wrote :

You know liberty had been EOL for long time. I don't think we would maintain it.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.