Autoscaling nested stack update incorrectly implemented

Bug #1360292 reported by Steven Hardy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Medium
Angus Salkeld

Bug Description

In a couple of places we've got this happening:

    updater = self.update_with_template(new_template,
                                                self._environment())
    updater.run_to_completion()
    self.check_update_complete(updater)

This means all update operations involving InstanceGroup and AutoScalingGroup resources will be serialized, since we're not following the correct pattern where we start the update in handle_update and poll for completion in check_create_complete.

This will become a more problematic issue when we move to decoupled nested stacks, since it means the parent stack will block on the nested stack update, when it could be doing something else and just periodically polling the nested stack state in check_update_complete instead

Revision history for this message
Steven Hardy (shardy) wrote :

Additional notes:

- stack-update initiated updates of InstanceGroup resource counts will block in the parent stack

- check_update_complete is only called once, so we don't actually wait for the update to complete when doing rolling updates etc.

Revision history for this message
Zane Bitter (zaneb) wrote :

Yep, this was originally only meant to happen when scaling was triggered from an alarm. Now that we're doing it during a rolling update, that's definitely bad for anything supposed to be happening in parallel.

Changed in heat:
status: New → Triaged
importance: Undecided → Medium
milestone: none → juno-3
Steven Hardy (shardy)
Changed in heat:
assignee: nobody → Steven Hardy (shardy)
Revision history for this message
Steven Hardy (shardy) wrote :

Oh wow, fixing this looks pretty hard :(

Revision history for this message
Steven Hardy (shardy) wrote :

Since fixing this looks hard/risky I'm going to defer to Kilo

Changed in heat:
milestone: juno-3 → next
Angus Salkeld (asalkeld)
Changed in heat:
assignee: Steven Hardy (shardy) → Angus Salkeld (asalkeld)
milestone: next → kilo-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/160595

Changed in heat:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/160595
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=f3e7fd5de217201777b70de182ab85c4542c7fc6
Submitter: Jenkins
Branch: master

commit f3e7fd5de217201777b70de182ab85c4542c7fc6
Author: Angus Salkeld <email address hidden>
Date: Thu Mar 5 10:49:50 2015 +1000

    Correct the check_update_completion logic in instance_group

    The automated resource update logic is incompatible with the up coming
    logic in decouple-nested, as it doesn't poll the check_update_complete
    correctly.

    Change-Id: I9c3d1f314ce7da3abda391ff2e0ab05d6c1d76ae
    Closes-bug: #1360292

Changed in heat:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/163743

Thierry Carrez (ttx)
Changed in heat:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in heat:
milestone: kilo-3 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.