Failed to migrate encrypted volume

[Lisa Li]

Openstack version:
From git.
stack@devstack1-hp:~/nova$ git log -n 3
commit 996c2f6f0543fa40fe34ca940bd56d8fd3f2e0f7
Merge: 4cf44ea e55d335
Author: Jenkins <email address hidden>
Date: Fri Nov 27 19:34:45 2015 +0000

    Merge "NFS setup for live-migration job"

Test steps:
1. Create an encrypted volume in Cinder.
2. Attach it to an VM.
3. Migrate the volume to another backend storage.

Expected result:
Migration succeeds.

Actual result:
Failed to rebase the device.

^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m self.volume_api.unreserve_volume(context, new_volume_id)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m six.reraise(self.type_, self.value, self.tb)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/nova/nova/compute/manager.py", line 4797, in _swap_volume
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m resize_to)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1209, in swap_volume
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m self._disconnect_volume(old_connection_info, disk_dev)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1068, in _disconnect_volume
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m driver.disconnect_volume(connection_info, disk_dev)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/nova/nova/virt/libvirt/volume/iscsi.py", line 93, in disconnect_volume
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m self.connector.disconnect_volume(connection_info['data'], None)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 254, in inner
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m return f(*args, **kwargs)

^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 254, in inner
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/os-brick/os_brick/initiator/connector.py", line 743, in disconnect_volume
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m self._disconnect_volume_iscsi(props)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/os-brick/os_brick/initiator/connector.py", line 764, in _disconnect_volume_iscsi
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m self._linuxscsi.wait_for_volume_removal(host_device)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/os-brick/os_brick/utils.py", line 54, in _wrapper
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m return r.call(f, *args, **kwargs)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 212, in call
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m raise attempt.get()
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 247, in get
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m six.reraise(self.value[0], self.value[1], self.value[2])
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 200, in call
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m File "/opt/stack/os-brick/os_brick/initiator/linuxscsi.py", line 82, in wait_for_volume_removal
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m volume_path=volume_path)
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00mVolumePathNotRemoved: Volume path /dev/disk/by-path/ip-10.0.2.15:3260-iscsi-iqn.2010-10.org.openstack:volume-f6dcb40c-0477-43a9-8a58-4dd27deb0580-lun-1 was not removed in time.
^[[01;31m2015-12-03 02:30:30.155 TRACE oslo_messaging.rpc.dispatcher ^[[01;35m^[[00m

Analysis:
As this is a encrypted volume, it needs to detach encryptors.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/252809

[Pawel Koniszewski (pawel-koniszewski)]

Volume migration to another backend is not related to nova's live migration functionality

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/252809
Reason: This review is > 4 weeks without comment, and is not mergable in it's current state. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Vishakha Agarwal (vishakha.agarwal)]

Hi all,

Is anyone working on this issue??

Thanks

[Lee Yarwood (lyarwood)]

This should be resolved as of https://review.openstack.org/#/c/460243/ AFAIK.