Temporary volumes not cleaned up after retype for some backends

Bug #1743008 reported by Carl Pecinovsky
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
In Progress
Undecided
IBM Storage

Bug Description

The scenario is retyping a volume from one backend to another. The issue seems to happen for drivers that are sensitive to looking up volumes by name on the backend and have implemented driver.update_migrated_volume() in order to rename the new target volume on the backend so that the original volume that gets retyped can be found properly. For example the IBM Storwize driver.

This works great when that driver is the target of the retype operation. It's update_migrated_volume() method is called by the cinder manager's migrate_volume_completion() flow and the retyped volume can then be looked up and managed properly by the driver.

However, if that type of driver owns the original (source) volume of the retype operation, then they never get called to update their backend volume so that it can be looked up via the temporary cinder volume record that gets created during the operation.

After the cross-driver migration completes, the volume data and metadata is swapped between the two volumes and the temporary volume gets deleted. Then the retype finishes.

In this problematic scenario, the temporary volume is not found on the backend (because it was not renamed to match anything) and the driver assumes it has already been deleted. It fails silently and the temporary volume continues to be left around on the source backend for the the retype operation.
Here is an example of the silent failure:

2017-10-17 06:23:14.916 5893 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-706d2a1d-10bc-4554-bfe2-8391b3e13938 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 e6c428a2cd104f89b75bd9e6d540aa0e - default default] Tried to delete non-existent vdisk volume-RHEL_data_vol_tmpLJANu_-0619a972-2560.
2017-10-17 06:23:14.929 5893 INFO cinder.volume.manager [req-706d2a1d-10bc-4554-bfe2-8391b3e13938 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 e6c428a2cd104f89b75bd9e6d540aa0e - default default] Deleted volume successfully.

Carl Pecinovsky (csky)
Changed in cinder:
assignee: nobody → Carl Pecinovsky (csky)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/533803

Changed in cinder:
status: New → In Progress
Revision history for this message
Sean McGinnis (sean-mcginnis) wrote :

Drivers should not be using names for volume identification.

tags: added: driver storwize
Revision history for this message
Carl Pecinovsky (csky) wrote :

Sean, I let the ibm storage driver team know about this and assigned it back there. Thanks.

Changed in cinder:
assignee: Carl Pecinovsky (csky) → IBM Storage (ibm-storage)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (master)

Change abandoned by Sean McGinnis (<email address hidden>) on branch: master
Review: https://review.openstack.org/533803

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.