Ophaned Volume Not Removed on Instance Delete

Bug #1344113 reported by Mat Lowery on 2014-07-18
This bug affects 5 people
Affects Status Importance Assigned to Milestone
OpenStack DBaaS (Trove)
Ali Asgar Adil

Bug Description

When the trove.taskmanager.models.FreshInstanceTasks#_create_server_volume_individually path is used, cinderclient is used to create the volume and then that volume is supplied in the block_device_mapping (BDM) in the novaclient call. delete_on_terminate is set to True in that BDM. However, in a scenario in which cinderclient succeeds and the subsequent novaclient fails (e.g. quota), the volume is not cleaned up since there is no nova server.

relevant code-path: https://github.com/openstack/trove/blob/1654031cc6b0fa28efb322717efabf2bebffabf8/trove/taskmanager/models.py#L583-L597

this is not good, because the quotas are rolled back, meaning there's a provisioned volume that's not accounted for.

Changed in trove:
status: New → Confirmed
Changed in trove:
status: Confirmed → Triaged
importance: Undecided → Medium
milestone: none → juno-3
milestone: juno-3 → ongoing
description: updated
Changed in trove:
milestone: ongoing → kilo-1
Changed in trove:
assignee: nobody → Auston McReynolds (amcrn)
summary: - volume is not deleted if nova call fails after cinder call
+ Ophaned Volume Not Removed on Instance Delete

Fix proposed to branch: master
Review: https://review.openstack.org/125550

Changed in trove:
status: Triaged → In Progress
Changed in trove:
milestone: kilo-1 → kilo-2
Changed in trove:
milestone: kilo-2 → kilo-3
Changed in trove:
milestone: kilo-3 → kilo-rc1
Changed in trove:
milestone: kilo-rc1 → liberty-1
Changed in trove:
milestone: liberty-1 → liberty-2
Changed in trove:
assignee: Auston McReynolds (amcrn) → Nikhil Manchanda (slicknik)
milestone: liberty-2 → liberty-3
Changed in trove:
milestone: liberty-3 → ongoing

Change abandoned by Craig Vyvial (<email address hidden>) on branch: master
Review: https://review.openstack.org/125550
Reason: Abandoning due to lack activity updating this patch. Feel free to open this patch back up. Thanks.

Matt Fischer (mfisch) wrote :

I would argue that this should be high. When this happens you can run out of quota which is not apparent in the trove logs and is confusing to customers.

Amrith Kumar (amrith) on 2016-05-06
Changed in trove:
importance: Medium → High

Fix proposed to branch: master
Review: https://review.openstack.org/334722

Changed in trove:
assignee: Nikhil Manchanda (slicknik) → Ali Asgar Adil (aadil-b)

Reviewed: https://review.openstack.org/334722
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=c3d670c2186e681d89cfa511984eff7255ef0826
Submitter: Jenkins
Branch: master

commit c3d670c2186e681d89cfa511984eff7255ef0826
Author: Ali Adil <email address hidden>
Date: Mon Jun 27 20:47:23 2016 -0400

    Ophaned Volume Not Removed on Instance Delete

    When an instance is created with quota issues, the cinder volume is
    created butthe nova instance fails to be created. Deleting the
    instance would fail todelete the volume. Check the status of the
    volume when deleting the instance, if the status of the volume is
    "available" delete the volume as it is not connected to any instance.

    Change-Id: Ie921a8ff2851e2d9d76a3c3836945c750f090c4e
    Closes-Bug: #1344113

Changed in trove:
status: In Progress → Fix Released

This issue was fixed in the openstack/trove development milestone.

This issue was fixed in the openstack/trove development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers