Reviewed: https://review.opendev.org/660280 Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=a053b1db0faa636cc226fca506a1180694b05e55 Submitter: Zuul Branch: stable/queens
commit a053b1db0faa636cc226fca506a1180694b05e55 Author: Hervé Beraud <email address hidden> Date: Fri May 3 00:55:56 2019 +0200
Fix switch connection destination when a rabbitmq cluster node disappear
In a clustered rabbitmq when a node disappears, we get a ConnectionRefusedError because the socket get disconnected.
The socket access yields a OSError because the heartbeat tries to reach an unreachable host (No route to host).
Catch these exceptions to ensure that we call ensure_connection for switching the connection destination.
POC is available at github.com:4383/rabbitmq-oslo_messging-error-poc
Example: $ git clone <email address hidden>:4383/rabbitmq-oslo_messging-error-poc $ cd rabbitmq-oslo_messging-error-poc $ python -m virtualenv . $ source bin/activate $ pip install -r requirements.txt $ sudo podman run -d --hostname my-rabbit --name rabbit rabbitmq:3 $ python poc.py $(sudo podman inspect rabbit | niet '.[0].NetworkSettings.IPAddress')
And in parallele in an another shell|tmux $ podman stop rabbit $ # observe the output of the poc.py script we now call ensure_connection
Now you can observe some output relative to the connection who is modified and not catched before these changes.
Related to: https://bugzilla.redhat.com/show_bug.cgi?id=1665399
Closes-Bug: #1828841
Change-Id: I9dc1644cac0e39eb11bf05f57bde77dcf6d42ed3 (cherry picked from commit 9d8b1430e5c081b081c0e3c0b5f12f744dc7809d)
Reviewed: https:/ /review. opendev. org/660280 /git.openstack. org/cgit/ openstack/ oslo.messaging/ commit/ ?id=a053b1db0fa a636cc226fca506 a1180694b05e55
Committed: https:/
Submitter: Zuul
Branch: stable/queens
commit a053b1db0faa636 cc226fca506a118 0694b05e55
Author: Hervé Beraud <email address hidden>
Date: Fri May 3 00:55:56 2019 +0200
Fix switch connection destination when a rabbitmq cluster node disappear
In a clustered rabbitmq when a node disappears, we get a efusedError because the socket get disconnected.
ConnectionR
The socket access yields a OSError because the heartbeat
tries to reach an unreachable host (No route to host).
Catch these exceptions to ensure that we call ensure_connection for switching
the connection destination.
POC is available at github. com:4383/ rabbitmq- oslo_messging- error-poc
Example: :4383/rabbitmq- oslo_messging- error-poc oslo_messging- error-poc ttings. IPAddress' )
$ git clone <email address hidden>
$ cd rabbitmq-
$ python -m virtualenv .
$ source bin/activate
$ pip install -r requirements.txt
$ sudo podman run -d --hostname my-rabbit --name rabbit rabbitmq:3
$ python poc.py $(sudo podman inspect rabbit | niet '.[0].NetworkSe
And in parallele in an another shell|tmux
$ podman stop rabbit
$ # observe the output of the poc.py script we now call ensure_connection
Now you can observe some output relative to the connection who is
modified and not catched before these changes.
Related to: https:/ /bugzilla. redhat. com/show_ bug.cgi? id=1665399
Closes-Bug: #1828841
Change-Id: I9dc1644cac0e39 eb11bf05f57bde7 7dcf6d42ed3 081c0e3c0b5f12f 744dc7809d)
(cherry picked from commit 9d8b1430e5c081b