IntegrityError when deleting pre-stored raw template

Bug #1626256 reported by Zane Bitter
14
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Low
Zane Bitter

Bug Description

When creating a nested stack and pre-storing the raw template, we use a try/finally block to ensure that if the stack creation was never started (e.g. because the template failed validation at the other end) that we would delete the raw template again rather than leave it lying around in the DB. However, this catches too broad a range of exceptions. If an exception such as GreenletExit occurs inside this block, then we will attempt to delete the raw template, resulting in a DB IntegrityError either there (because the template id is already referenced by a stack) or when trying to store the stack (because the template is gone) - whichever happens second.

Prior to https://review.openstack.org/#/c/369827/ being merged, this could be reproduced fairly easily using the steps from bug 1499669. There's probably no way to trigger it any more with that change merged, but it would be nice to tidy this up.

Changed in heat:
status: Triaged → In Progress
Zane Bitter (zaneb)
Changed in heat:
milestone: none → ocata-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/374340
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=55c21c18e6455515c93f6f6222e7dd820c45ebd5
Submitter: Jenkins
Branch: master

commit 55c21c18e6455515c93f6f6222e7dd820c45ebd5
Author: Zane Bitter <email address hidden>
Date: Wed Sep 21 14:57:22 2016 -0400

    Don't delete RawTemplate if it referenced by a stack

    When creating a nested stack and pre-storing the raw template, we
    previously used a try/finally block to ensure that if the stack creation
    was never started (e.g. because the template failed validation at the other
    end) that we would delete the raw template again rather than leave it lying
    around in the DB. However, this catches too broad a range of exceptions. If
    an exception such as GreenletExit occurs inside this block, then we will
    attempt to delete the raw template, resulting in a DB IntegrityError either
    here (because the template id is already referenced by a stack) or when
    trying to store the stack (because the template is gone).

    Resolve this by only doing the cleanup on HeatExceptions, which indicates
    that the other engine has bailed out without storing the stack.

    Change-Id: Ie24a82b7ce16281c72d4eb66cca8683248628b46
    Closes-Bug: #1626256

Changed in heat:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 8.0.0.0b1

This issue was fixed in the openstack/heat 8.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/501592

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/newton)

Reviewed: https://review.openstack.org/501592
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=c64e5d5db9011790c5258780127494a7ecc25758
Submitter: Jenkins
Branch: stable/newton

commit c64e5d5db9011790c5258780127494a7ecc25758
Author: Zane Bitter <email address hidden>
Date: Wed Sep 21 14:57:22 2016 -0400

    Don't delete RawTemplate if it referenced by a stack

    When creating a nested stack and pre-storing the raw template, we
    previously used a try/finally block to ensure that if the stack creation
    was never started (e.g. because the template failed validation at the other
    end) that we would delete the raw template again rather than leave it lying
    around in the DB. However, this catches too broad a range of exceptions. If
    an exception such as GreenletExit occurs inside this block, then we will
    attempt to delete the raw template, resulting in a DB IntegrityError either
    here (because the template id is already referenced by a stack) or when
    trying to store the stack (because the template is gone).

    Resolve this by only doing the cleanup on HeatExceptions, which indicates
    that the other engine has bailed out without storing the stack.

    Change-Id: Ie24a82b7ce16281c72d4eb66cca8683248628b46
    Closes-Bug: #1626256
    (cherry picked from commit 55c21c18e6455515c93f6f6222e7dd820c45ebd5)

tags: added: in-stable-newton
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.