Failure after the update of a suspended stack

Bug #1234242 reported by Simon Pasquier
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Medium
Angus Salkeld
Havana
Fix Released
Medium
Unassigned

Bug Description

How to reproduce:
- Create a stack with OS::Heat::InstanceGroup of N instances
- Suspend the stack
- Update the stack with instances number = N+1
=> the status of the stack gets UPDATE_FAILED
=> when trying to resume the stack, it fails too as only a SUSPEND_COMPLETE stack can be resumed.

It would probably best to discard stack updates when in SUSPEND_COMPLETE state.

Heat engine logs (after the update):
2013-10-02 17:03:24.690 22867 ERROR heat.engine.resource [-] update InstanceGroup "Computes" : 'NoneType' object has no attribute 't'
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource Traceback (most recent call last):
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource File "/usr/local/heat/local/lib/python2.7/site-packages/heat/engine/resource.py", line 420, in update
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource result = self.handle_update(after, tmpl_diff, prop_diff)
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource File "/usr/local/heat/lib/python2.7/site-packages/heat/engine/resources/autoscaling.py", line 175, in handle_update
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource self.resize(int(self.properties['Size']))
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource File "/usr/local/heat/lib/python2.7/site-packages/heat/engine/resources/autoscaling.py", line 217, in resize
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource new_template = self._create_template(new_capacity)
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource File "/usr/local/heat/lib/python2.7/site-packages/heat/engine/resources/autoscaling.py", line 200, in _create_template
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource instance_definition = copy.deepcopy(conf.t)
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource AttributeError: 'NoneType' object has no attribute 't'
2013-10-02 17:03:24.690 22867 TRACE heat.engine.resource

Revision history for this message
Steven Hardy (shardy) wrote :

I agree, I think the easiest fix is just to not allow stacks to be updated if they're suspended.

Angus Salkeld (asalkeld)
Changed in heat:
assignee: nobody → Angus Salkeld (asalkeld)
importance: Undecided → Medium
status: New → Triaged
milestone: none → icehouse-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/51094

Changed in heat:
status: Triaged → In Progress
Revision history for this message
Michał Kamiński (michal-kaminski) wrote :

But what about updating stack when suspending is in progress ?

Revision history for this message
Steven Hardy (shardy) wrote :

> But what about updating stack when suspending is in progress ?

That has been addressed as part of the fix:

https://review.openstack.org/#/c/51094/

tags: added: havana-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/51094
Committed: http://github.com/openstack/heat/commit/8e5229ca453178f395804933b530ffce1e5dc31e
Submitter: Jenkins
Branch: master

commit 8e5229ca453178f395804933b530ffce1e5dc31e
Author: Angus Salkeld <email address hidden>
Date: Wed Oct 16 11:21:08 2013 +1100

    Do not attempt a stack update when it is suspended

    And in any state (inprogress, complete, failed).

    Closes-bug: #1234242
    Change-Id: Ic0bccfa3d3d98cb41d0f5228d8cdc164c5c1fd32

Changed in heat:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/56833

Thierry Carrez (ttx)
Changed in heat:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/havana)

Reviewed: https://review.openstack.org/56833
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=ec4ed57627e8127978125bcbed1cf9fc03a991e0
Submitter: Jenkins
Branch: stable/havana

commit ec4ed57627e8127978125bcbed1cf9fc03a991e0
Author: Angus Salkeld <email address hidden>
Date: Wed Oct 16 11:21:08 2013 +1100

    Do not attempt a stack update when it is suspended

    And in any state (inprogress, complete, failed).

    Closes-bug: #1234242
    Change-Id: Ic0bccfa3d3d98cb41d0f5228d8cdc164c5c1fd32

Thierry Carrez (ttx)
Changed in heat:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.