[zmq] Implement outgoing queue in green version of zmq driver

Bug #1497315 reported by Oleksii Zamiatin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.messaging
Fix Released
Undecided
Oleksii Zamiatin

Bug Description

Despite zmq claims DEALER socket as reliable one:

    according to the RFC http://rfc.zeromq.org/spec:28

        * "SHALL NOT discard messages that it cannot queue."

    but also:

        * " SHALL block on sending, or return a suitable error, when it has no available peers."

we definitely face blocking of the main thread (especially if using eventlet it's critical issue) when trying to send a message before consumer appears. To avoid this and keep reliable messaging we need to provide outgoing queue(s) where to put all outgoing messages.

This outgoing queue (or outgoing poller which contains the queue) should be similar to incoming GreenPoller which does the same but for incoming messages. We probably need to refactor-rename existing pollers that way they can take consistent place in green-pollers hierarchy.

Messages should be removed from outgoing queue when their delivery has been acknowledged by consumer side.

Tags: zmq
description: updated
Changed in oslo.messaging:
status: New → Confirmed
Changed in oslo.messaging:
assignee: nobody → Oleksii Zamiatin (ozamiatin)
status: Confirmed → In Progress
summary: - Implement outgoing queue in green version of zmq driver
+ [zmq] Implement outgoing queue in green version of zmq driver
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (master)

Fix proposed to branch: master
Review: https://review.openstack.org/227913

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (master)

Reviewed: https://review.openstack.org/227913
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=3067dbd1984c984108707ab27d2694f80db9f1d4
Submitter: Jenkins
Branch: master

commit 3067dbd1984c984108707ab27d2694f80db9f1d4
Author: Oleksii Zamiatin <email address hidden>
Date: Fri Sep 25 20:34:39 2015 +0300

    Non-blocking outgoing queue was implemented

    The main issue with eventlet.green.zmq is that libzmq as a C-library
    is completely monkey-patch unfriendly. So any blocking call inside
    the native library makes calling process stuck. We can't avoid this
    actually in an absolutely normal situation when a client appears
    earlier than listener we have all client process get stuck until listener
    raised. If the listener for example is also blocked awaiting for some
    other service to appear we have a chain of locks which may occasionally
    result in a dead-lock. The other situation with Notifier is quite similar.

    For that reason zmq-broker was restored, but now it serves as an outgoing
    queue on a client side. Servers remained the same dynamically port-binded.
    Now all clients can still use green-zmq, but presence of the broker-queue
    on a host guarantees that green threads will never blocked in a client
    because all messages will wait their listeners inside the broker queue.
    The broker process's modules are not monkey-patched, they make use of native
    threading and native zmq.

    Possibility to run without broker also remains. The option zmq_use_broker
    introduced for that reason.

    Closes-Bug: #1497315

    Change-Id: I786b100fd6ee1cf4b99139db0ca044d358d36345

Changed in oslo.messaging:
status: In Progress → Fix Committed
Changed in oslo.messaging:
milestone: none → 2.6.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.