cinder volume deletions-in-progress lead to stack deletion errors

[Joe D'Andrea]

Scenario: A stack containing a significantly-sized OS::Cinder::Volume has been queued for deletion via stack-delete (not out-of-band).

Consider the case where this stack deletion was interrupted (e.g., an error). Meanwhile, the cinder volume still happens to be in a deleting state.

The stack-delete request is made once again, and a new error is thrown:

http://paste.openstack.org/show/101829/

I propose that Heat should gracefully ignore the volume and not throw an error, as it is already being deleted.

[Rakesh H S (rh-s)]

I was able to reproduce this issue

[Joe D'Andrea (jdandrea)]

I have a basic patch (which would be my first ever). I will review the procedure for supplying that.

In the meantime, is Bug 1298350 related? I just noticed it.

https://bugs.launchpad.net/heat/+bug/1298350

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/118374

[Rakesh H S (rh-s)]

Hi Joe,

Bug 1298350 is not related. Anyways we are able to reproduce the issue on the latest code base, so we need to fix this.
I had taken this bug in launchpad(check the "Assigned to" flag) and already worked on it. I have pushed one patchset to gerrit and working on unit tests for the same.

I am sorry but you are a bit late.
One suggestion: you need first assign the bug to your name and then start working on the fix.

Regards,
Rakesh

[Joe D'Andrea (jdandrea)]

Good to hear!

No worries about being late. I'm glad that you could address it. (I was unsure if I should wait for it to be triaged before assigning it to my name. I'll get used to the process before long, I'm sure.)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/118374
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=6aca29e05bb0cfe469ed17f4ab9bb9351b2f108a
Submitter: Jenkins
Branch: master

commit 6aca29e05bb0cfe469ed17f4ab9bb9351b2f108a
Author: Rakesh H S <email address hidden>
Date: Tue Sep 2 20:06:53 2014 +0530

    fixes stack deletion failure, when vol in deleting

    when an delete operation is requested on a stack, and the volume in stack
    is already in 'deleting' state, heat fails to delete the stack and goes to
    DELETE_FAILED state instantly.

    Instead of going to DELETE_FAILED state instantly, heat needs to safely
    wait for the volume to complete its deletion process, so that heat can
    then go ahead and complete stack deletion successfully.

    Closes-Bug: #1362708

    Change-Id: I088eab90bee4bc804abe9e505fd61b17b266def1