Replaced resources can be left UPDATE_IN_PROGRESS

[Zane Bitter]

If a resource requests a replacement, either by raising UpdateReplace the handle_update() method or by returning True from needs_replace_with_tmpl_diff(), it is left in the UPDATE_IN_PROGRESS state. This is fine if the update completes normally (since the resource being replaced will be removed before the stack update completes).

However, if something fails and the stack is not rolled back, either because a subsequent resource update/create fails or the enginer dies, this leaves the resource stuck in an IN_PROGRESS state (it may be in the main stack or the backup stack). And this state is not reset when a new heat-engine starts up because the stack containing it will be correctly marked as FAILED.

It would be better to move these resources to the UPDATE_COMPLETE state, with the resource_status_reason indicating that they are to be replaced.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/296080

[Zane Bitter (zaneb)]

Thanks to huangtianhua for pointing out in https://review.openstack.org/#/c/296080/1/heat/engine/resource.py that there is code to move the resource back to UPDATE_COMPLETE, so the scenario described above will generally not happen. However, this code is not inside a try/finally or a context manager, so there remains a small window in which an unexpected exception could leave the resource in the wrong state.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/296080
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=1ecf93c19ca868e29fd7b60f3a35c5610adb6365
Submitter: Jenkins
Branch: master

commit 1ecf93c19ca868e29fd7b60f3a35c5610adb6365
Author: Zane Bitter <email address hidden>
Date: Mon Mar 28 16:26:20 2016 -0400

    Mark resources UPDATE_COMPLETE when they request replacement

    When a resource plugin requests for a resource to be replaced (e.g. by
    raising UpdateReplace in handle_update()), mark the resource as
    UPDATE_COMPLETE with the reason, instead of leaving it as
    UPDATE_IN_PROGRESS.

    Change-Id: I966b04908e6739bbadadeb20ba94ca8322c95f7d
    Closes-Bug: #1560688

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b1 development milestone.