Race condition in eventletutils Event

[Zane Bitter]

Victor Stinner reported[1] a race condition in the class in the eventletutils module that is designed to provide an eventlet Event with an interface compatible with the threading module's Event class:

Is there a race condition on this line in a greenthread A is blocked on the wait, but another greenthread B calls clear() and then set()? B calls self._event.send(), but A is waiting on a different eventlet Event which is no longer used by the oslo.service Event...

eventlet.event.Event implements a reset() method but it's documented as broken and suggest to create a new Event object:

        # this is kind of a misfeature and doesn't work perfectly well,
        # it's better to create a new event rather than reset an old one
# removing documentation so that we don't get new use cases for it

https://github.com/eventlet/eventlet/blob/master/eventlet/event.py#L53

If there is a race condition, maybe we need something like:

def clear(self):
    old_event = self._event
    self._set = False
    self._event = event.Event()
    # Unblock any greenthreads blocked in wait()
    old_event.send(True)

def wait():
    event = self._event
    while True:
        event.wait()
        if event is not self._event:
            # clear has been called, retry
        break
    ...

[1] https://review.openstack.org/#/c/611807/5/oslo_service/loopingcall.py@101

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/618482
Committed: https://git.openstack.org/cgit/openstack/oslo.utils/commit/?id=cc8b51e1e16f6bdc7d6c0e571e2002e70cde098d
Submitter: Zuul
Branch: master

commit cc8b51e1e16f6bdc7d6c0e571e2002e70cde098d
Author: Zane Bitter <email address hidden>
Date: Wed Nov 28 15:50:12 2018 -0500

    Fix race condition in eventletutils Event

    The threading-compatible eventlet Event class has a race condition on
    the wait method. If greenthread A is blocked on the wait, but another
    greenthread B calls clear() and then set(), B calls self._event.send(),
    but A is waiting on a different eventlet Event which is no longer used
    by the oslo.service Event...

    To resolve this, when clearing an Event trigger the underlying eventlet
    Event immediately, then have the wait() method resume waiting on the new
    eventlet Event.

    Change-Id: I81579e2977bb965a5398a2cb4e3e24f5671e856a
    Co-Authored-By: Victor Stinner <email address hidden>
    Co-Authored-By: Hervé Beraud <email address hidden>
    Closes-Bug: #1805706

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.utils 3.39.1 release.