Heat stack is not always deleted

[Stan Lagun]

Sometimes Heat stack doesn't get deleted after environment delete.

In engine's log there is
2016-02-22 22:48:20.303 19814 WARNING murano.dsl.executor [-] Muted exception during execution of .destroy on <io.murano.Environment/0.0.0 80936f96c54b48b5a860eec290e62614 (140043876743824)>: [exceptions.EnvironmentError]: Unexpected stack state UPDATE_COMPLETE: Stack UPDATE completed successfully
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor Traceback (most recent call last):
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 239, in _cleanup
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor method.invoke(self, obj, (), {}, None)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/murano_method.py", line 140, in invoke
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor self, this, context, args, kwargs, skip_stub)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 71, in invoke_method
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor method, this, context, args, kwargs, skip_stub=skip_stub)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 87, in _invoke_method
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor yaql_engine, method_context, this.real_this)(*args, **kwargs)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/usr/local/lib/python2.7/dist-packages/yaql/language/specs.py", line 60, in func
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor return self.get_delegate(receiver, engine, context, args, kwargs)()
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/usr/local/lib/python2.7/dist-packages/yaql/language/specs.py", line 341, in func
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor six.iteritems(keyword_args)))
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/yaql_integration.py", line 240, in payload
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor executor, __receiver, args, kwargs, __context, True)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/murano_method.py", line 140, in invoke
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor self, this, context, args, kwargs, skip_stub)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 71, in invoke_method
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor method, this, context, args, kwargs, skip_stub=skip_stub)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 129, in _invoke_method
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor result = call()
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/executor.py", line 124, in call
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor else method.body.execute(context))
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/macros.py", line 58, in execute
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor super(MethodBlock, self).execute(new_context)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/macros.py", line 40, in execute
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor expr.execute(context)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor File "/opt/stack/murano/murano/dsl/expressions.py", line 78, in execute
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor e, context)
2016-02-22 22:48:20.303 19814 ERROR murano.dsl.executor MuranoPlException: [exceptions.EnvironmentError]: Unexpected stack state UPDATE_COMPLETE: Stack UPDATE completed successfully

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/283327

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/283329

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/283309
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=25416ac6d4dc6fe0b3db1403474b604819e1fcd4
Submitter: Jenkins
Branch: master

commit 25416ac6d4dc6fe0b3db1403474b604819e1fcd4
Author: Stan Lagun <email address hidden>
Date: Tue Feb 23 02:46:18 2016 +0300

    Heat stack was not always deleted

    HeatStack.delete() method didn't check stack timestamps
    before deletion. As a result it couldn't correctly detect the
    fact that delete already started. So it could get UPDATE_COMPLETE
    status from last operation instead and fail because it wasn't expected.

    Closes-Bug: #1548567
    Change-Id: I66130f826c6f72562a7c36920b0decae63adf5e0

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/283327
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=0ab82f77c3d0ecdaa4dea59a70d2bc6e625bb16f
Submitter: Jenkins
Branch: stable/liberty

commit 0ab82f77c3d0ecdaa4dea59a70d2bc6e625bb16f
Author: Stan Lagun <email address hidden>
Date: Tue Feb 23 03:37:55 2016 +0300

    Heat stack was not always deleted

    HeatStack.delete() method didn't check stack timestamps
    before deletion. As a result it couldn't correctly detect the
    fact that delete already started. So it could get UPDATE_COMPLETE
    status from last operation instead and fail because it wasn't expected.

    Closes-Bug: #1548567
    Change-Id: I66130f826c6f72562a7c36920b0decae63adf5e0

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/murano 2.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/283329
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=cafdcb9d481c71e2d6abe512f2ff5ed19e797d50
Submitter: Jenkins
Branch: stable/kilo

commit cafdcb9d481c71e2d6abe512f2ff5ed19e797d50
Author: Stan Lagun <email address hidden>
Date: Tue Feb 23 03:37:55 2016 +0300

    Heat stack was not always deleted

    HeatStack.delete() method didn't check stack timestamps
    before deletion. As a result it couldn't correctly detect the
    fact that delete already started. So it could get UPDATE_COMPLETE
    status from last operation instead and fail because it wasn't expected.

    Closes-Bug: #1548567
    Change-Id: I66130f826c6f72562a7c36920b0decae63adf5e0

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/murano 2015.1.1 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/murano 1.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/murano 2015.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/murano 1.0.3 release.