in-used volume can't be migrated for InvalidType

[xueyingchen(zte)]

Reproduction steps：
1. Boot instance from image (create a new volume) , Instance spawned successfully.
cli command:
nova boot --flavor 1 --block-device id=b171c0ba-0532-4e38-be4d-05c291e88403,source=image,dest=volume,bootindex=0,size=20 --nic net-id=e499cc2e-c6bb-4d70-9fe8-616f5399bf49 vm1

In the cmd，b171c0ba-0532-4e38-be4d-05c291e88403 is an image.

2. Migrate/Retype the volume which created by step 1.
cli command:
cinder migrate <volume-uuid> <dest-volume-host>
Or cmd:
cinder retype <volume-uuid> <another-type> on-demand

3.The volume migrate/retype failed

4.Get error infomation in nova-compute.log :

2016-03-07 16:17:48.718 9817 WARNING nova.virt.libvirt.volume [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 0
2016-03-07 16:17:49.503 9817 ERROR nova.compute.manager [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] Failed to swap volume 333df485-406c-4c51-92d1-5ba0aebfeb0d for 241abd7b-31fb-4763-86e7-3eda59744f09
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] Traceback (most recent call last):
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5914, in _swap_volume
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] resize_to)
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1238, in swap_volume
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] driver_bdm = driver_block_device.DriverVolumeBlockDevice(bdm)
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 102, in __init__
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] self._transform()
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 204, in _transform
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] raise _InvalidType
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] _InvalidType
2016-03-07 16:17:49.503 9817 TRACE nova.compute.manager [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab]
2016-03-07 16:17:49.771 9817 INFO nova.compute.manager [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] swap_volume: calling Cinder terminate_connection for 241abd7b-31fb-4763-86e7-3eda59744f09
2016-03-07 16:17:50.775 9817 INFO nova.compute.manager [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] swap_volume:calling Cinder terminate_connection end for: 241abd7b-31fb-4763-86e7-3eda59744f09
2016-03-07 16:17:51.239 9817 INFO nova.compute.manager [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] [instance: 303abfcb-b78b-4891-848f-b1f490ac69ab] swap_volume: Cinder migrate_volume_completion returned: {u'save_volume_id': u'333df485-406c-4c51-92d1-5ba0aebfeb0d'}
2016-03-07 16:17:51.392 9817 INFO nova.scheduler.client.report [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] Compute_service record updated for ('2C5_10_DELL05', '2C5_10_DELL05')
2016-03-07 16:17:51.394 9817 ERROR oslo_messaging.rpc.dispatcher [req-8039f9e7-9d4d-48dc-9e2a-dd6073a954b0 8b34e1ab75024fcba0ea69a6fd0937c3 181a578bc97642f2b9e153bec622f130 - - -] Exception during message handling:
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8708, in swap_volume
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher new_volume_id)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher payload)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 379, in decorated_function
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 350, in decorated_function
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 407, in decorated_function
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 395, in decorated_function
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5974, in swap_volume
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher new_volume_id)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5933, in _swap_volume
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, new_volume_id)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5914, in _swap_volume
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher resize_to)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1238, in swap_volume
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher driver_bdm = driver_block_device.DriverVolumeBlockDevice(bdm)
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 102, in __init__
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher self._transform()
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 204, in _transform
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher raise _InvalidType
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher _InvalidType
2016-03-07 16:17:51.394 9817 TRACE oslo_messaging.rpc.dispatcher

[weiweigu (gu-weiwei)]

boot instance from image (create a new volume), and in the block_device_mapping table of nova db, the source_type is image, not volume.
When migrating volume(in-use status), will checking whether the volume type is valid, the _valid_source is volume. but
 in the block_device_mapping table of nova db, the source_type is image.

/nova/virt/block_device.py:
class DriverVolumeBlockDevice(DriverBlockDevice):
    _legacy_fields = set(['connection_info', 'mount_device',
                          'delete_on_termination'])
    _new_fields = set(['guest_format', 'device_type',
                       'disk_bus', 'boot_index'])
    _fields = _legacy_fields | _new_fields

    _valid_source = 'volume'
    _valid_destination = 'volume'

    _proxy_as_attr = set(['volume_size', 'volume_id'])
    _update_on_save = {'disk_bus': None,
                       'device_name': 'mount_device',
                       'device_type': None}

    def _transform(self):
        if (not self._bdm_obj.source_type == self._valid_source
                or not self._bdm_obj.destination_type ==
                self._valid_destination):
            LOG.info(_LI('%s,%s,%s,%s') % (self._bdm_obj.source_type,
                          self._valid_source,self._bdm_obj.destination_type,self._valid_destination))
            raise _InvalidType

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/314455

[OpenStack Infra (hudson-openstack)]

Change abandoned by Weiwei Gu (<email address hidden>) on branch: master
Review: https://review.openstack.org/314455
Reason: abandon

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/315864

[Eli Qiao (taget-9)]

@weiwei, thanks for update the bug, I reproduced it by:

nova volume-update test_bdm $(old_boot_volume_id) $(new_boot_volume_id)

this should be same logic with 'cinder migrate/retype'.

2016-07-07 11:08:29.744 DEBUG oslo_messaging._drivers.amqpdriver [req-44373ed6-f6b2-4ff0-b6a3-59f0304f2b56 admin admin] CALL msg_id: f9dce7e1
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in force_reraise
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 156, in decorated_function
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 215, in decorated_function
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 221, in __exit__
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server self.force_reraise()
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in force_reraise
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 203, in decorated_function
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 4871, in swap_volume
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server resize_to)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 4823, in _swap_volume
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server self.volume_api.unreserve_volume(context, new_volume_id)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 221, in __exit__
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server self.force_reraise()
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in force_reraise
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 4804, in _swap_volume
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server resize_to)
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1309, in swap_volume
2016-07-07 11:08:29.795 TRACE oslo_messaging.rpc.server driver_bdm = driver_block_device.DriverVolumeB...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/315864
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ab1d40149bba06f80dde8e8645ab3ab5f16abf02
Submitter: Jenkins
Branch: master

commit ab1d40149bba06f80dde8e8645ab3ab5f16abf02
Author: weiweigu <email address hidden>
Date: Fri May 13 13:05:58 2016 +0800

    migration volume failed for invalid type

    Boot an instance from image (create a new volume), then migrate this
    volume. When migrating, source_type and destination_type will be
    checked for validity at function
    nova.virt.block_device.DriverVolumeBlockDevice._transform().
    The valid source is volume. But source_type is
    not volume, is image in the block_device_mapping table. It causes
    migration failure.
    Actually source_type may be image snapshot and volume. At
    swap_volume, it should call the convert_volume, but not
    DriverVolumeBlockDevice.

    Change-Id: I36b2e2aef3345f244c05ae94225c91938450a749
    Closes-Bug: #1554859

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/nova 14.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/433791

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/433791