oslo.service binding same port for multiple processes

[venkata anil]

In oslo.service, backdoor_port [1] allows processes spawned by oslo.service for a service to listen on unique port [2] if we provide a port range. Code block [2] i.e

def _listen(host, start_port, end_port, listen_func):
    try_port = start_port
    while True:
        try:
            return listen_func((host, try_port))
        except socket.error as exc:
            if (exc.errno != errno.EADDRINUSE or
               try_port >= end_port):
                raise
            try_port += 1

will always try with first port in the provided range and will try the next only if the listen fails. But because of commit [3] in eventlet (which went in version v0.20.0), which supports listening on same port, our code will always listen on same port for all the processes. Openstack cinder team has already noticed this issue long back [4] for their CI failures and proposed this change [5], i.e replace eventlet.listen() with directly calling socket.listen [6]

We should also do the same change in oslo.service i.e [6].

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.bind(("127.0.0.1", 0))
    sock.listen(50)

[1] https://github.com/openstack/oslo.service/blob/master/oslo_service/_options.py#L26
[2] https://github.com/openstack/oslo.service/blob/master/oslo_service/eventlet_backdoor.py#L124-L133
[3] https://github.com/eventlet/eventlet/commit/f9a3074a3b75f17f76cc04a693dc48a367b99861
[4] https://github.com/eventlet/eventlet/issues/411
[5] https://review.openstack.org/#/c/463876/2
[6] https://review.openstack.org/#/c/463876/2/test/__init__.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/627952

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/627952
Committed: https://git.openstack.org/cgit/openstack/oslo.service/commit/?id=811650783d32accd2b49f907ddc520955146962c
Submitter: Zuul
Branch: master

commit 811650783d32accd2b49f907ddc520955146962c
Author: venkata anil <email address hidden>
Date: Wed Jan 2 05:11:12 2019 -0500

    Avoid eventlet_backdoor listing on same port

    Oslo.service is binding to same port when we provide a port range
    as eventlet is internally setting SO_REUSEPORT flag (starting from
    eventlet version v0.20). And there is a flag (reuse_port)
    introduced in v0.22 to give control to user to avoid SO_REUSEPORT.
    In this patch, first we try passing reuse_port=False, if this fails
    then directly open socket and listen instead of eventlist.listen.

    Closes-Bug: #1810280
    Change-Id: Idc842acc7e430199c76fe12785b0bf0e7a58e121

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.service 1.36.0 release.