po 22. 5. 2023 v 15:10 odesílatel Andrew Bogott <email address hidden>
napsal:
> this seems similar to https://bugs.launchpad.net/charm-rabbitmq-
> server/+bug/1993149 which is now fixed with
> https://review.opendev.org/c/openstack/oslo.messaging/+/866617
>
> If you don't want to wait for the backport, you can implement a similar
> fix in config by setting kombu_reconnect_delay to 0.5.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2019978
>
> Title:
> oslo_messaging kombu strategy round-robin not working correctly
>
> Status in oslo.messaging:
> New
>
> Bug description:
> Hi,
>
> We were doing some HA tests against our openstack cluster and we were
> testing what will happen if we turn off one rabbitmq from 3 node
> rabbitmq cluster and found that default oslo.messaging
> kombu_failover_strategy = round-robin introduced in
>
> https://github.com/openstack/oslo.messaging/commit/6ae46796a61fc97467450b5bdd51dc6a0c86f9f4
> probably not working as expected.
>
> We turned off 10.157.106.71 and clients didn't reconnect. If I grepped
> occurences in logs for this rabbitmq server, i found that it is always
> trying that host which is turned off.
>
>
> root@controller0:/home/ubuntu# grep -Ri '2023-05-17.*Trying again in'
> /var/log/kolla | awk '{print $11}' | sort | uniq -c
> 5 10.157.106.136:5672
> 12 10.157.106.6:5672
> 50381 10.157.106.71:5672
>
> root@controller13:/home/ubuntu# grep -Ri '2023-05-17.*Trying again in'
> /var/log/kolla | awk '{print 11}' | sort | uniq -c
> 2 -]
> 6 10.157.106.136:5672
> 4 10.157.106.6:5672
> 41996 10.157.106.71:5672
>
> I was also checking TCP SYN via netstat and I saw every time it was
> trying to connect to rabbitmq server which was down.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/oslo.messaging/+bug/2019978/+subscriptions
>
>
Hi Andrew,
yeah, you are right, we already confirmed that mentioned patch is fixing
issue, but unit tests in yoga are failing ...do you know why ?
Thanks
Michal Arbet
Openstack Engineer
Ultimum Technologies a.s.
Na Poříčí 1047/26, 11000 Praha 1
Czech Republic
+420 604 228 897 /ultimum. io <https:/ /ultimum. io/>*
<email address hidden>
*https:/
LinkedIn <https:/ /www.linkedin. com/company/ ultimum- technologies> | Twitter /twitter. com/ultimumtech> | Facebook /www.facebook. com/ultimumtech nologies/ timeline>
<https:/
<https:/
po 22. 5. 2023 v 15:10 odesílatel Andrew Bogott <email address hidden>
napsal:
> this seems similar to https:/ /bugs.launchpad .net/charm- rabbitmq- /review. opendev. org/c/openstack /oslo.messaging /+/866617 _delay to 0.5. /bugs.launchpad .net/bugs/ 2019978 strategy = round-robin introduced in /github. com/openstack/ oslo.messaging/ commit/ 6ae46796a61fc97 467450b5bdd51dc 6a0c86f9f4 0:/home/ ubuntu# grep -Ri '2023-05-17.*Trying again in' 13:/home/ ubuntu# grep -Ri '2023-05-17.*Trying again in' /bugs.launchpad .net/oslo. messaging/ +bug/2019978/ +subscriptions
> server/+bug/1993149 which is now fixed with
> https:/
>
> If you don't want to wait for the backport, you can implement a similar
> fix in config by setting kombu_reconnect
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> oslo_messaging kombu strategy round-robin not working correctly
>
> Status in oslo.messaging:
> New
>
> Bug description:
> Hi,
>
> We were doing some HA tests against our openstack cluster and we were
> testing what will happen if we turn off one rabbitmq from 3 node
> rabbitmq cluster and found that default oslo.messaging
> kombu_failover_
>
> https:/
> probably not working as expected.
>
> We turned off 10.157.106.71 and clients didn't reconnect. If I grepped
> occurences in logs for this rabbitmq server, i found that it is always
> trying that host which is turned off.
>
>
> root@controller
> /var/log/kolla | awk '{print $11}' | sort | uniq -c
> 5 10.157.106.136:5672
> 12 10.157.106.6:5672
> 50381 10.157.106.71:5672
>
> root@controller
> /var/log/kolla | awk '{print 11}' | sort | uniq -c
> 2 -]
> 6 10.157.106.136:5672
> 4 10.157.106.6:5672
> 41996 10.157.106.71:5672
>
> I was also checking TCP SYN via netstat and I saw every time it was
> trying to connect to rabbitmq server which was down.
>
> To manage notifications about this bug go to:
> https:/
>
>