Heat should retry on DBConnectionError when releasing lock
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Heat |
Triaged
|
High
|
Unassigned |
Bug Description
When Heat gets a DBConnectionError because the database is down, this is generally treated as an error and whatever write we were trying to do does not occur. This is probably the right behaviour in many cases, but there are circumstances where it can be problematic.
Specifically, when releasing a stack lock at the end of an operation, Heat should probably retry releasing the lock until the database comes back online. If it does not and simply lets the exception bubble up and kill the thread, then the engine not only retains the lock but remains alive so that no other engine can steal it. (This can be worked around comparatively easily by restarted the engine that holds the lock, but it's not at all obvious when it happens that that's what you need to do.)
Changed in heat: | |
milestone: | newton-1 → newton-2 |
Changed in heat: | |
milestone: | newton-2 → newton-3 |
Changed in heat: | |
milestone: | newton-3 → next |
More details in https:/ /bugzilla. redhat. com/show_ bug.cgi? id=1308517# c4