rollback-on-failure is not deleting the sync points

Bug #1618155 reported by Anant Patil
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
High
Anant Patil

Bug Description

Steps to reproduce:
1. create a stack with http://paste.openstack.org/show/564629/
2. Update the above stack with rollback option with template: http://paste.openstack.org/show/564630/

In second template a resource is marked to fail, so the rollback happens but in DB the sync points remain.

mysql> select * from sync_point;
+--------------------------------------+--------------------------------------+-----------+------------+--------------------------------------+-------------------------------------------+---------------------+---------------------+
| entity_id | traversal_id | is_update | atomic_key | stack_id | input_data | created_at | updated_at |
+--------------------------------------+--------------------------------------+-----------+------------+--------------------------------------+-------------------------------------------+---------------------+---------------------+
| 1 | db1ef9e9-79d2-4231-9566-f8891a1c928f | 0 | 1 | 2b2d256b-573b-4946-ac89-7deac47716de | {"input_data": {"tuple:(1, True)": null}} | 2016-08-29 17:38:13 | 2016-08-29 17:38:16 |
| 1 | db1ef9e9-79d2-4231-9566-f8891a1c928f | 1 | 0 | 2b2d256b-573b-4946-ac89-7deac47716de | {} | 2016-08-29 17:38:13 | NULL |
| 2 | db1ef9e9-79d2-4231-9566-f8891a1c928f | 1 | 0 | 2b2d256b-573b-4946-ac89-7deac47716de | {} | 2016-08-29 17:38:13 | NULL |
| 2b2d256b-573b-4946-ac89-7deac47716de | db1ef9e9-79d2-4231-9566-f8891a1c928f | 1 | 0 | 2b2d256b-573b-4946-ac89-7deac47716de | {} | 2016-08-29 17:38:13 | NULL |
+--------------------------------------+--------------------------------------+-----------+------------+--------------------------------------+-------------------------------------------+---------------------+---------------------+
4 rows in set (0.00 sec)

Anant Patil (ananta)
Changed in heat:
assignee: nobody → Anant Patil (ananta)
Revision history for this message
Anant Patil (ananta) wrote :

The problem is caused due to commit 084d0eb20f6636b63721eb3c5a9db08858d05ec5 where we update the current traversal in state_set, and by doing so we loose the previous traversal. Since we loose the previous traversal ID, there is no way to go back and delete it. The code at https://github.com/openstack/heat/blob/master/heat/engine/check_resource.py#L75 is causing the issue.

My worry is that we do state_set at various places and implicitly setting the current traversal is not a good idea. The solution is to probably just update the current traversal as another instruction/method so that we know what action needs to be taken explicitly when we do so.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/362301

Changed in heat:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on heat (master)

Change abandoned by Anant Patil (<email address hidden>) on branch: master
Review: https://review.openstack.org/362301
Reason: Fixing in another patch instead of reverting the older one.

Zane Bitter (zaneb)
Changed in heat:
milestone: none → newton-rc1
importance: Undecided → High
Thomas Herve (therve)
Changed in heat:
milestone: newton-rc1 → none
milestone: none → nexton-rc2
Revision history for this message
Anant Patil (ananta) wrote :
Zane Bitter (zaneb)
tags: added: newton-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/362346
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=2e281df4280de5fdf5a56827d38845090e720392
Submitter: Jenkins
Branch: master

commit 2e281df4280de5fdf5a56827d38845090e720392
Author: Anant Patil <email address hidden>
Date: Wed Sep 14 16:59:55 2016 +0530

    Fix sync point delete

    When a resource failed, the stack state was set to FAILED and current
    traversal was set to emoty string. The actual traversal was lost and
    there was no way to delete the sync points belonging to the actual
    traversal.

    This change keeps the current traversal when you do a state set, so that
    later you can delete the sync points belonging to it. Also, the current
    traversal is set to empty when the stack has failed and there is no need
    to rollback.

    Closes-Bug: #1618155

    Change-Id: Iec3922af92b70b0628fb94b7b2d597247e6d42c4

Changed in heat:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/372729

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/newton)

Reviewed: https://review.openstack.org/372729
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=c7fe0ba5ee46dd97c5d07f8c45df6c382f24306c
Submitter: Jenkins
Branch: stable/newton

commit c7fe0ba5ee46dd97c5d07f8c45df6c382f24306c
Author: Anant Patil <email address hidden>
Date: Wed Sep 14 16:59:55 2016 +0530

    Fix sync point delete

    When a resource failed, the stack state was set to FAILED and current
    traversal was set to emoty string. The actual traversal was lost and
    there was no way to delete the sync points belonging to the actual
    traversal.

    This change keeps the current traversal when you do a state set, so that
    later you can delete the sync points belonging to it. Also, the current
    traversal is set to empty when the stack has failed and there is no need
    to rollback.

    Closes-Bug: #1618155

    Change-Id: Iec3922af92b70b0628fb94b7b2d597247e6d42c4
    (cherry picked from commit 2e281df4280de5fdf5a56827d38845090e720392)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 7.0.0.0rc2

This issue was fixed in the openstack/heat 7.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 7.0.0

This issue was fixed in the openstack/heat 7.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 8.0.0.0b1

This issue was fixed in the openstack/heat 8.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.