Can not delete stack if volume go to error_deleting state

[Sergey Kraynev]

Some times volumes may go to state error_deleting.
There are some ways to delete them: via force delete or manually from db.

On Heat side it leads to stack which can not be deleted, because it's constantly meet error that:

Error Resource deletion with status error_deleting.

Should we handle it in the Heat?

[Steven Hardy (shardy)]

I don't see how we can handle it in heat, if it's a persistent failure, the best we can do is ignore it.

A possible option would be a --ignore-error option or similar which forces traversal of the whole stack even when we encounter an error (which is not sure to succeed because there may be a dependency on the thing which can't be deleted).

IMO this is a cinder bug, so it's probably best to debug the root cause of the delete volume error.

[Steven Hardy (shardy)]

Oh, hmm, there is cinder force-delete isn't there, yes we could have an option which allows heat to use that :)

[Tatiana Kholkina (tlashchova)]

We can't handle it in the Heat because force-delete works only with admin account.

[Sergey Kraynev (skraynev)]

Tetiana: may be I missed something, but we already have some resources available only for admin. So I do not see any reasons do not it here.
Also the main idea it's not only force delete. The main aim is to guarantee, that stack will be deleted in such case (We can handle such volume status, but I still prefer delete volume, if we can do it)

[Pavlo Shchelokovskyy (pshchelo)]

Sergey, the point is that force-delete would not work if user is not an admin, and we really do not want to move generic volume resource to be admin-only. IMO that would be not appropriate to report the stack as successfully deleted but leave an actual (billable) resource behind, stack must report DELETE_FAILED anyway. The only solution I see then is to try to delete as much of the dependency graph as possible accumulating errors in the progress, and then report the failed status. Is it what you have in mind?

[Sergey Kraynev (skraynev)]

Pavlo: I'd prefer to leave one undeleted resource and report that stack was deleted with such issue.
The main case, why I want exactly it, is when you use f.e. rally for testing you can not delete some stacks and it delays all other tests during timeout. Also if you have big deployment you may got a lot of such failed stacks.

Approach suggested by you make sense, but I still want to solve issues above. Any thoughts about compromise?

[Song Li (lisong-cruise)]

IMO, as the volume is created by heat, if the volume deleted failed, it is reasonable for heat to delete failed.

But, I think there might be something can be done to improve for your case, for example.
when the volume deletion failed, heat will jump the deleltion and continue to delete other resources. Of course, the stack deletion result is failed with the infomation why stack deletion failed. Customer can manually delete the volume and then delete the stack.