[zmq] ZMQ driver eats too many TCP sockets

[Alexey Yelistratov]

Oslo.messaging 0mq driver eats too many sockets on controller nodes and is not yet ready to use on large clusters. Current scale limit is 400 nodes. Each additional compute node needs ~100 new connections to the controllers.

MOS 8.0 + openstack Liberty + python-oslo.messaging 2.5.0-1~u14.04+mos16

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/287094
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=cc1cb3032177ec80d84cff409c71ce3204122f53
Submitter: Jenkins
Branch: master

commit cc1cb3032177ec80d84cff409c71ce3204122f53
Author: Oleksii Zamiatin <email address hidden>
Date: Wed Mar 2 12:41:52 2016 +0200

    [zmq] Reduce number of connections

    In this change 'use_router_proxy' option was added
    to switch between direct connections and proxy.

    Proxy was reimplemented and splitted onto two
    types of proxies:

        * PUBLISHER proxy for fanout pattern
        * ROUTER proxy for direct messaging

    Each type of proxy is configured over command line
    argument --type.

    Deployment guide was updated accordingly to the change.

    Change-Id: If36e9c26e2a8ebe622cfa7e9f2a07b1a69aabe34
    Closes-Bug: #1555007

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/308653
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=b5955b6ca938bfccfeefb37062bb5ff81c8895a4
Submitter: Jenkins
Branch: master

commit b5955b6ca938bfccfeefb37062bb5ff81c8895a4
Author: ozamiatin <email address hidden>
Date: Thu Apr 21 01:06:39 2016 +0300

    [zmq] Redesign router proxy

    In this change router was redesigned in a way most
    appropriate for routing concept of zmq.ROUTER socket.

    DEALER(cli)-ROUTER(proxy)-DEALER(srv) instead of
    DEALER-ROUTER-DEALER-ROUTER (3 layers instead of 4)

    The main reason is to use zmq.DEALER identity in message
    routing. For this reason DealerConsumer was introduced
    server-side. RouterConsumer is left for peer-to-peer
    DEALER-ROUTER deployment option.

    Also handled assertions in receive-methods in order
    to not stop server when received message with wrong format.

    Change-Id: If25edf500fa8d220d4233bb13d67121824e841c6
    Closes-Bug: #1558601
    Related-Bug: #1555007

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/oslo.messaging 5.0.0 release.