False positives of *_FAILED status in heat_integrationtests

[Sirushti Murugesan]

When running heat_integrationtests in parallel, this happened intermittently.

2015-04-14 12:23:47.092 | 2015-04-14 12:23:47.082 | heat_integrationtests.functional.test_remote_stack.RemoteStackTest.test_stack_suspend_resume
2015-04-14 12:23:47.094 | 2015-04-14 12:23:47.084 | --------------------------------------------------------------------------------------------
2015-04-14 12:23:47.095 | 2015-04-14 12:23:47.085 |
2015-04-14 12:23:47.097 | 2015-04-14 12:23:47.087 | Captured traceback:
2015-04-14 12:23:47.099 | 2015-04-14 12:23:47.088 | ~~~~~~~~~~~~~~~~~~~
2015-04-14 12:23:47.100 | 2015-04-14 12:23:47.090 | Traceback (most recent call last):
2015-04-14 12:23:47.101 | 2015-04-14 12:23:47.091 | File "/opt/stack/new/heat/heat_integrationtests/functional/test_remote_stack.py", line 143, in test_stack_suspend_resume
2015-04-14 12:23:47.103 | 2015-04-14 12:23:47.093 | self.stack_suspend(stack_id)
2015-04-14 12:23:47.105 | 2015-04-14 12:23:47.094 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 437, in stack_suspend
2015-04-14 12:23:47.107 | 2015-04-14 12:23:47.096 | 'SUSPEND_COMPLETE')
2015-04-14 12:23:47.108 | 2015-04-14 12:23:47.098 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 202, in _wait_for_all_resource_status
2015-04-14 12:23:47.110 | 2015-04-14 12:23:47.099 | success_on_not_found=success_on_not_found)
2015-04-14 12:23:47.111 | 2015-04-14 12:23:47.101 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 231, in _wait_for_resource_status
2015-04-14 12:23:47.113 | 2015-04-14 12:23:47.102 | resource_status_reason=res.resource_status_reason)
2015-04-14 12:23:47.115 | 2015-04-14 12:23:47.105 | heat_integrationtests.common.exceptions.StackResourceBuildErrorException: Resource my_stack in stack RemoteStackTest-428049341/cd4f3830-ce39-4f35-b045-985e72881f42 is in SUSPEND_FAILED status due to 'ResourceUnknownStatus: Resource failed - Unknown status CREATE_COMPLETE due to "Resource action mismatch detected: expected=SUSPEND actual=CREATE"'

So, while we wait for the stack to reach $(ACTION)_COMPLETE, we also do a check against the stack status to see if it failed but we don't check for the action explicitly which will result in the checking of possibly a stale status therefore leading to $(failure_traceback).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/175161
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=c5794aa8d244d90092e1479001c07585501782ac
Submitter: Jenkins
Branch: master

commit c5794aa8d244d90092e1479001c07585501782ac
Author: Sirushti Murugesan <email address hidden>
Date: Sat Apr 18 23:30:59 2015 +0530

    Prevent false positives of *_FAILED status

    Closes-Bug: #1446098

    Change-Id: I9bf182a1c9d83df1b6fcffa5713eceb5118aba6b