During testing of Murano we faced with the following issue:
Heat stack is handing on in UPDATE_IN_PROGRESS state or DELETE_IN_PROGRESS case when we are trying to create and update the stack in series.
The case is the following:
1. Create the stack with murano_first template (using neutron router in devstack).
2. Update the stack with murano_second template using m1.express flavor( flavor RAM value should be enough to raise No valid host found in Nova)
The stack is failing because of no valid host found error.
3. Update the failed stack with murano_third (using neutron public net as floating network).
The stack is failing again because of no valid host found error.
4. Delete the stack.
Expected result: the stack has been deleted successfully.
Actual result: the stack is hanging on in DELETE_IN_PROGRESS
This is specialty of Murano that it sends update requests in a bunch and doesn't check that the previous update was successful.
But we definitely need to treat properly such kind of cases.
When I looked into the heat log I found that because of some reason heat generated Invalid Template Reference error.
After deep analysis it turned out that Heat does not create backup resources properly in the case below.
That happens because of following:
0. Server (..ks41) in the second template has a reference to Neutron Port.
1. When Heat is trying to update the server first time everything is working correctly.
1. When Heat is trying to update the failed server (...ks41) second time it is trying to re-create it.
2. During re-creation Heat is backing up failed server independently of stack-update result and saves that resource to DB. (but the Port was not saved because there were no updates for it)
3. When Heat is trying to delete stack we have got an error because the server has a reference to the port but the port does not exist in the template.
So I suppose we should prohibit backing up of failed resource in the case above.