commit dc8da5bb194d0a544fb8065aa9a4c7484f605715
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 10:45:12 2017 -0700
Use "poll" or "selects" Eventlet hub for all Swift daemons.
Previously, Swift's WSGI servers, the object replicator, and the
object reconstructor were setting Eventlet's hub to either "poll" or
"selects", depending on availability. Other daemons were letting
Eventlet use its default hub, which is "epoll".
In any daemons that fork, we really don't want to use epoll. Epoll
instances end up shared between the parent and all children, and you
get some awful messes when file descriptors are shared.
Here's an example where two processes are trying to wait on the same
file descriptor using the same epoll instance, and everything goes
wrong:
This primarily affects the container updater and object updater since
they fork. I've decided to change the hub for all Swift daemons so
that we don't add multiprocessing support to some other daemon someday
and suffer through this same bug again.
This problem was made more apparent by commit 6d16079, which made our
logging mutex use file descriptors. However, it could have struck on
any shared file descriptor on which a read or write returned EAGAIN.
Reviewed: https:/ /review. openstack. org/511550 /git.openstack. org/cgit/ openstack/ swift/commit/ ?id=dc8da5bb194 d0a544fb8065aa9 a4c7484f605715
Committed: https:/
Submitter: Zuul
Branch: master
commit dc8da5bb194d0a5 44fb8065aa9a4c7 484f605715
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 10:45:12 2017 -0700
Use "poll" or "selects" Eventlet hub for all Swift daemons.
Previously, Swift's WSGI servers, the object replicator, and the
object reconstructor were setting Eventlet's hub to either "poll" or
"selects", depending on availability. Other daemons were letting
Eventlet use its default hub, which is "epoll".
In any daemons that fork, we really don't want to use epoll. Epoll
instances end up shared between the parent and all children, and you
get some awful messes when file descriptors are shared.
Here's an example where two processes are trying to wait on the same
file descriptor using the same epoll instance, and everything goes
wrong:
[proc A] epoll_ctl(6, EPOLL_CTL_ADD, 3, ...) = 0
[proc B] epoll_ctl(6, EPOLL_CTL_ADD, 3, ...) = -1 EEXIST (File exists)
[proc B] epoll_wait(6, ...) = 1
[proc B] epoll_ctl(6, EPOLL_CTL_DEL, 3, ...) = 0
[proc A] epoll_wait(6, ...)
This primarily affects the container updater and object updater since
they fork. I've decided to change the hub for all Swift daemons so
that we don't add multiprocessing support to some other daemon someday
and suffer through this same bug again.
This problem was made more apparent by commit 6d16079, which made our
logging mutex use file descriptors. However, it could have struck on
any shared file descriptor on which a read or write returned EAGAIN.
Change-Id: Ic2c1178ac918c8 8b0b901e581eb4f ab3b2666cfe
Closes-Bug: 1722951