test_volume_migrate_attached failed on Dell SC driver

Bug #1826840 reported by Sam Wan on 2019-04-29
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
Undecided
Unassigned

Bug Description

tempest test case 'tempest.scenario.test_volume_migrate_attached.TestVolumeMigrateRetypeAttached.test_volume_migrate_attached' failed on Dell SC driver for both SC and iSCSI.

FC error log: http://publiclogs.emc.com/97/655597/1/check/DellEMC_SC_FC/531f32c/DellEMC_SC_FC/8/logs/screen-c-vol.txt.gz
======
Apr 25 02:56:59.673119 dellscfc cinder-volume[17715]: DEBUG cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-b5af6fbe-6099-4af7-a1c0-9aad40dc0703 req-00f7af15-6233-4329-b1da-0efd50488139 tempest-TestVolumeMigrateRetypeAttached-1555718387 None] post: ApiConnection/Logout data: {} {{(pid=17902) post /opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_api.py:250}}
Apr 25 02:56:59.680336 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager [req-b5af6fbe-6099-4af7-a1c0-9aad40dc0703 req-00f7af15-6233-4329-b1da-0efd50488139 tempest-TestVolumeMigrateRetypeAttached-1555718387 None] Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.: cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.
Apr 25 02:56:59.680812 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 25 02:56:59.680948 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4579, in _connection_terminate
Apr 25 02:56:59.681094 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager force=force)
Apr 25 02:56:59.681247 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager File "/opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_fc.py", line 311, in terminate_connection
Apr 25 02:56:59.681388 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager data=_('Terminate connection unable to connect to backend.'))
Apr 25 02:56:59.681524 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.
Apr 25 02:56:59.681655 dellscfc cinder-volume[17715]: ERROR cinder.volume.manager
Apr 25 02:56:59.681802 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server [req-b5af6fbe-6099-4af7-a1c0-9aad40dc0703 req-00f7af15-6233-4329-b1da-0efd50488139 tempest-TestVolumeMigrateRetypeAttached-1555718387 None] Exception during message handling: cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.
Apr 25 02:56:59.682142 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Apr 25 02:56:59.682273 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4579, in _connection_terminate
Apr 25 02:56:59.682407 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server force=force)
Apr 25 02:56:59.682537 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_fc.py", line 311, in terminate_connection
Apr 25 02:56:59.682670 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server data=_('Terminate connection unable to connect to backend.'))
Apr 25 02:56:59.682809 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.
Apr 25 02:56:59.682945 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server
Apr 25 02:56:59.683079 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
Apr 25 02:56:59.683211 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server
Apr 25 02:56:59.683341 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Apr 25 02:56:59.683470 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming
Apr 25 02:56:59.683599 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Apr 25 02:56:59.683732 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
Apr 25 02:56:59.683873 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Apr 25 02:56:59.684011 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
Apr 25 02:56:59.684144 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Apr 25 02:56:59.684283 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4615, in attachment_delete
Apr 25 02:56:59.684417 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server self._do_attachment_delete(context, vref, attachment_ref)
Apr 25 02:56:59.684547 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4622, in _do_attachment_delete
Apr 25 02:56:59.684682 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server attachment)
Apr 25 02:56:59.684815 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4587, in _connection_terminate
Apr 25 02:56:59.684952 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg)
Apr 25 02:56:59.685088 dellscfc cinder-volume[17715]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: Terminate connection unable to connect to backend.
======

iSCSI error log: http://publiclogs.emc.com/97/655597/1/check/DellEMC_SC_iSCSI/d333490/DellEMC_SC_iSCSI/8/logs/screen-c-vol.txt.gz
======
Apr 25 04:07:33.857119 dellscslave cinder-volume[6422]: DEBUG cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-7a2fc635-a0d9-41d8-a725-09676011a1c7 req-7be344f1-cc79-4df7-a114-682a2801fa1f tempest-TestVolumeMigrateRetypeAttached-64622798 None] post: ApiConnection/Logout data: {} {{(pid=6577) post /opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_api.py:250}}
Apr 25 04:07:33.882531 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager [req-7a2fc635-a0d9-41d8-a725-09676011a1c7 req-7be344f1-cc79-4df7-a114-682a2801fa1f tempest-TestVolumeMigrateRetypeAttached-64622798 None] Terminate volume connection failed: Terminate connection failed: cinder.exception.VolumeBackendAPIException: Terminate connection failed
Apr 25 04:07:33.882703 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 25 04:07:33.882822 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4579, in _connection_terminate
Apr 25 04:07:33.882922 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager force=force)
Apr 25 04:07:33.883024 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager File "/opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_iscsi.py", line 305, in terminate_connection
Apr 25 04:07:33.883119 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager _('Terminate connection failed'))
Apr 25 04:07:33.883215 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager cinder.exception.VolumeBackendAPIException: Terminate connection failed
Apr 25 04:07:33.883311 dellscslave cinder-volume[6422]: ERROR cinder.volume.manager
Apr 25 04:07:33.885106 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server [req-7a2fc635-a0d9-41d8-a725-09676011a1c7 req-7be344f1-cc79-4df7-a114-682a2801fa1f tempest-TestVolumeMigrateRetypeAttached-64622798 None] Exception during message handling: cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Terminate connection failed
Apr 25 04:07:33.885265 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Apr 25 04:07:33.885480 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4579, in _connection_terminate
Apr 25 04:07:33.885671 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server force=force)
Apr 25 04:07:33.885984 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/drivers/dell_emc/sc/storagecenter_iscsi.py", line 305, in terminate_connection
Apr 25 04:07:33.886231 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server _('Terminate connection failed'))
Apr 25 04:07:33.886382 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeBackendAPIException: Terminate connection failed
Apr 25 04:07:33.886483 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server
Apr 25 04:07:33.886581 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
Apr 25 04:07:33.886682 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server
Apr 25 04:07:33.886776 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Apr 25 04:07:33.887365 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming
Apr 25 04:07:33.887469 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Apr 25 04:07:33.887581 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
Apr 25 04:07:33.887683 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Apr 25 04:07:33.887799 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
Apr 25 04:07:33.887908 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Apr 25 04:07:33.888163 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4615, in attachment_delete
Apr 25 04:07:33.888293 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server self._do_attachment_delete(context, vref, attachment_ref)
Apr 25 04:07:33.888400 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4622, in _do_attachment_delete
Apr 25 04:07:33.888577 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server attachment)
Apr 25 04:07:33.888678 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/cinder/cinder/volume/manager.py", line 4587, in _connection_terminate
Apr 25 04:07:33.888777 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg)
Apr 25 04:07:33.888874 dellscslave cinder-volume[6422]: ERROR oslo_messaging.rpc.server cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Terminate connection failed
======

Sam Wan (sam-wan) on 2019-04-29
tags: added: dell drivers sc
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers