Event listeners die unexpectedly

[Lucian Petrut]

PyMI subscribes to event providers, setting a callback that will be invoked when an event arrives. When called, the event watcher will block until an event is retrieved. A threading.Event is used internally by PyMI for the event waiting workflow.

The issue is that the threading module will be monkey patched by eventlet. The event callback is performed from a different thread, so attempting to set the event for which the event watcher waits will some times raise the "error: cannot switch to a different thread" greenlet exception, thus breaking the event listener.

This issue can be fixed at the os-win level, without having PyMI to know about eventlet. We can make sure that PyMI uses the unpatched threading module, also ensuring that the blocking calls are run in a different thread, so that we don't block the calling thread, preventing other greenthreads from continuing theri execution. eventlet.tpool can be used to easily achieve this.

Trace: http://paste.openstack.org/show/494805/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/304086
Committed: https://git.openstack.org/cgit/openstack/os-win/commit/?id=53a8d6d0b3381e6df2823257fa1aa1665077cea4
Submitter: Jenkins
Branch: master

commit 53a8d6d0b3381e6df2823257fa1aa1665077cea4
Author: Lucian Petrut <email address hidden>
Date: Mon Apr 11 14:45:44 2016 +0300

    Fix event listeners

    PyMI subscribes to event providers, setting a callback that will be
    invoked when an event arrives. When called, the event watcher will
    block until an event is retrieved. A threading.Event is used
    internally by PyMI for the event waiting workflow.

    The issue is that the threading module will be monkey patched by
    eventlet. The event callback is performed from a different thread, so
    attempting to set the event for which the event watcher waits will
    some times raise the "error: cannot switch to a different thread"
    greenlet exception, thus breaking the event listener.

    This issue can be fixed at the os-win level, without PyMI having to
    know about eventlet monkey patching. We can make sure that PyMI uses
    the unpatched threading module, also ensuring that the blocking calls
    are run in a different thread, so that we don't block the calling
    thread. eventlet.tpool can be used to easily achieve this.

    Closes-Bug: #1568824

    Change-Id: I9be6c1a1b144bdf565f25517c52439a5fb10cb51

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/307042

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/308174

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/307785
Committed: https://git.openstack.org/cgit/openstack/os-win/commit/?id=356b6cca37fd90c9ae449cf4bb0d2cddf480a34b
Submitter: Jenkins
Branch: master

commit 356b6cca37fd90c9ae449cf4bb0d2cddf480a34b
Author: Lucian Petrut <email address hidden>
Date: Tue Apr 19 15:17:59 2016 +0300

    Events: use tpool only if thread module is patched

    At the moment, we use eventlet.tpool when waiting for events
    even if the 'thread' module is not monkey patched, which causes
    locking issues.

    This patch changes this, checking if the 'thread' module is monkey
    patched, otherwise calling the method waiting for events directly.

    Related-Bug: #1568824

    Change-Id: Ib752bfa2acc9f6b47996f6e949aadf20903c849b

[OpenStack Infra (hudson-openstack)]

Change abandoned by Petrut Lucian (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/307042
Reason: Duplicate of https://review.openstack.org/#/c/308174/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/308174
Committed: https://git.openstack.org/cgit/openstack/os-win/commit/?id=37ac7ffd4d5a106f020704ecc331868da1354505
Submitter: Jenkins
Branch: stable/mitaka

commit 37ac7ffd4d5a106f020704ecc331868da1354505
Author: Lucian Petrut <email address hidden>
Date: Mon Apr 11 14:45:44 2016 +0300

    Fix event listeners

    PyMI subscribes to event providers, setting a callback that will be
    invoked when an event arrives. When called, the event watcher will
    block until an event is retrieved. A threading.Event is used
    internally by PyMI for the event waiting workflow.

    The issue is that the threading module will be monkey patched by
    eventlet. The event callback is performed from a different thread, so
    attempting to set the event for which the event watcher waits will
    some times raise the "error: cannot switch to a different thread"
    greenlet exception, thus breaking the event listener.

    This issue can be fixed at the os-win level, without PyMI having to
    know about eventlet monkey patching. We can make sure that PyMI uses
    the unpatched threading module, also ensuring that the blocking calls
    are run in a different thread, so that we don't block the calling
    thread. eventlet.tpool can be used to easily achieve this.

    Closes-Bug: #1568824

    Change-Id: I9be6c1a1b144bdf565f25517c52439a5fb10cb51
    (cherry picked from commit 53a8d6d0b3381e6df2823257fa1aa1665077cea4)

    Events: use tpool only if thread module is patched

    At the moment, we use eventlet.tpool when waiting for events
    even if the 'thread' module is not monkey patched, which causes
    locking issues.

    This patch changes this, checking if the 'thread' module is monkey
    patched, otherwise calling the method waiting for events directly.

    Related-Bug: #1568824

    Change-Id: Ib752bfa2acc9f6b47996f6e949aadf20903c849b
    (cherry-picked from commit 356b6cca37fd90c9ae449cf4bb0d2cddf480a34b)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/os-win 0.4.2 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/os-win 1.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-win 0.4.2 release.