Container updater may be stuck and not make progress
Bug #1722951 reported by
Timur Alperovich
This bug affects 5 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
High
|
Samuel Merritt | ||
Ubuntu Cloud Archive |
Fix Released
|
High
|
Unassigned | ||
Ocata |
Triaged
|
High
|
Unassigned | ||
Pike |
Triaged
|
High
|
Unassigned | ||
Queens |
Fix Released
|
High
|
Unassigned | ||
Rocky |
Fix Released
|
High
|
Unassigned | ||
swift (Ubuntu) |
Fix Released
|
High
|
Unassigned |
Bug Description
Container updater may be stuck and not make further progress. This was reported after the "PipeMutex" primitive was added to fix a deadlock in the logging layer. The bug can be reproduced with the attached python script.
The observed behavior is that after some time the container stats are no longer updated and strace for the daemons shows that they are all stuck in "epoll_wait".
Changed in swift: | |
assignee: | nobody → Samuel Merritt (torgomatic) |
Changed in cloud-archive: | |
status: | New → Triaged |
status: | Triaged → Fix Released |
importance: | Undecided → High |
Changed in swift (Ubuntu): | |
status: | New → Fix Released |
importance: | Undecided → High |
To post a comment you must log in.
The bug is real, but isn't PipeMutex's fault. We're letting eventlet pick the default hub sometimes. Eventlet chooses the epoll hub, and epoll doesn't play nicely with forking. (It's possible to use epoll and fork and still be correct, but eventlet doesn't make it so.)