attachment_create api leaks reserved attachments

Bug #1929678 reported by Felix Huettner
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
In Progress
High
Felix Huettner

Bug Description

Hi everyone,

the attachment_create api has the possibility to create volume attachments with and without a already specified connector. If no connector is specified the api will create a volume attachment db entry with the status "reserved" and put the volume to the "reserved" state. If a connector is specified then first the api creates the "reserved" information as above but then calls cinder-volume to actually combine the attachment with the connector leaving a "in-use" volume afterwards.

If we now assume that cinder-volume failes during the above step we will have a "reserved" volume attachment in the database and the volume will be in "reserved" state. This looks as follows:
* the volume is in "reserved"
* a volume-attachment in "reserved" state exists

This does not prevent the caller from trying again to attach the volume to the same instance as the "_attachment_reserve" method just creates an additional volume attachment record. If we assume cinder-volume has no issues this time (because it recovered in any way from the previous error) we now have the following state:
* the volume is in "in-use"
* a volume-attachment with the connector information exists and is "in-use"
* the old volume-attachment in "reserved" from the first step still exists

If the user now detaches the volume again cinder will clean up the currently active attachment, but the old "reserved" attachment still exists. Cinder will therfor put the volume in "reserved" state as well. This then looks as follows:
* the volume is in "reserved"
* the volume-attachment with the connector information is marked as deleted
* the old volume-attachment in "reserved" from the first step still exists

At this point the user requires admin intervention to attach the volume again to another instance. Deleting the volume is however possible.

The above can especially happen when there is a issue with cinder-volume during the _pre_live_migration of nova. Then nova will create the above reserved attachments that will not be cleaned up later.

My proposed solution would be to delete the attachment in the first step again if the call to "attachment_update" fails.

description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/793136

Changed in cinder:
status: New → In Progress
Changed in cinder:
assignee: nobody → Felix Huettner (felix.huettner)
Changed in cinder:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (master)

Change abandoned by "Felix Huettner <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/793136
Reason: Abandoning this as i currently have no time to work on it. If anyone is interested feel free to pick it up

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.