rabbitmq cluster does not work with dashes in the hostname

Bug #1540234 reported by weiyu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Michał Jastrzębski
Mitaka
Fix Released
Critical
weiyu

Bug Description

rabbitmq cluster is not work

bash-4.2$ rabbitmqctl cluster_status
Cluster status of node 'rabbit@openstack-2' ...
[{nodes,[{disc,['rabbit@openstack-2']}]},
 {running_nodes,['rabbit@openstack-2']},
 {cluster_name,<<"rabbit@openstack-2">>},
 {partitions,[]}]

bash-4.2$ cat /etc/rabbitmq/rabbitmq_clusterer.config
[
  {version, 1},
  {nodes, [
    {rabbit@openstack-1, disc},
    {rabbit@openstack-2, disc},
    {rabbit@openstack-3, disc}
  ]},
  {gospel, {node, rabbit@openstack-1}}
].

bash-4.2$ rpm -q rabbitmq-server
rabbitmq-server-3.5.5-3.noarch
bash-4.2$

bash-4.2$ rabbitmq-plugins list
 Configured: E = explicitly enabled; e = implicitly enabled
 | Status: * = running on rabbit@openstack-1
 |/
[e*] amqp_client 3.5.5
[ ] cowboy 0.5.0-rmq3.5.5-git4b93c2d
[e*] mochiweb 2.7.0-rmq3.5.5-git680dba8
[ ] rabbitmq_amqp1_0 3.5.5
[ ] rabbitmq_auth_backend_ldap 3.5.5
[ ] rabbitmq_auth_mechanism_ssl 3.5.5
[E*] rabbitmq_clusterer 3.5.x-189b3a81
[ ] rabbitmq_consistent_hash_exchange 3.5.5
[ ] rabbitmq_federation 3.5.5
[ ] rabbitmq_federation_management 3.5.5
[E*] rabbitmq_management 3.5.5
[e*] rabbitmq_management_agent 3.5.5
[ ] rabbitmq_management_visualiser 3.5.5
[ ] rabbitmq_mqtt 3.5.5
[ ] rabbitmq_shovel 3.5.5
[ ] rabbitmq_shovel_management 3.5.5
[ ] rabbitmq_stomp 3.5.5
[ ] rabbitmq_test 3.5.5
[ ] rabbitmq_tracing 3.5.5
[e*] rabbitmq_web_dispatch 3.5.5
[ ] rabbitmq_web_stomp 3.5.5
[ ] rabbitmq_web_stomp_examples 3.5.5
[ ] sockjs 0.3.4-rmq3.5.5-git3132eb9
[e*] webmachine 1.10.3-rmq3.5.5-gite9359c7

weiyu (weiyu)
description: updated
Revision history for this message
weiyu (weiyu) wrote :

bash-4.2$ cat /etc/rabbitmq/rabbitmq_clusterer.config
[
  {version, 1},
  {nodes, [
    {'rabbit@openstack-1', disc},
    {'rabbit@openstack-2', disc},
    {'rabbit@openstack-3', disc}
  ]},
  {gospel, {node, 'rabbit@openstack-1'}}
].

rabbitmq cluster is work

------------------------------------------------------------------------------------------
bash-4.2$ cat /etc/rabbitmq/rabbitmq_clusterer.config
[
  {version, 1},
  {nodes, [
    {rabbit@openstack-1, disc},
    {rabbit@openstack-2, disc},
    {rabbit@openstack-3, disc}
  ]},
  {gospel, {node, rabbit@openstack-1}}
].
bash-4.2$
rabbitmq cluster is not work

----------------------------------------------------------------------------------------
bash-4.2$ cat /etc/rabbitmq/rabbitmq_clusterer.config
[
  {version, 1},
  {nodes, [
    {rabbit@openstack1, disc},
    {rabbit@openstack2, disc},
    {rabbit@openstack3, disc}
  ]},
  {gospel, {node, rabbit@openstack-1}}
].
bash-4.2$
rabbitmq cluster is work

so , when a '-' in hostname, rabbitmq cluster will not work

weiyu (weiyu)
Changed in kolla:
assignee: nobody → weiyu (weiyu)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/275111
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=04992a3de0a181a658cae2561a3a1adce313bbf1
Submitter: Jenkins
Branch: master

commit 04992a3de0a181a658cae2561a3a1adce313bbf1
Author: weiyu <email address hidden>
Date: Tue Feb 2 18:37:44 2016 +0800

    fix miss rabbitmq config when hostname include special character

    Change-Id: I46db4d3547b120d1ccd87a48e31b699bdd1aa1a0
    Closes-Bug: #1540234

Changed in kolla:
status: New → Fix Released
Sam Yaple (s8m)
Changed in kolla:
status: Fix Released → Confirmed
Revision history for this message
Sam Yaple (s8m) wrote : Re: rabbitmq cluster is not work

This patch needs to be reverted as it breaks clusterer on ubuntu. Also, the behavior you are describing I cannot reproduce and I have a - in my hostname. The initial config is correct and clusterer should not be quoted like this, there is another issue at play

====================

Working on ubuntu:

[
  {version, 1},
  {nodes, [
    {rabbit@y-work-laptop01, disc}
  ]},
  {gospel, {node, rabbit@y-work-laptop01}}
].

====================

Broken on ubuntu:

[
  {version, 1},
  {nodes, [
    {'rabbit@y-work-laptop01', disc}
  ]},
  {gospel, {node, 'rabbit@y-work-laptop01'}}
].

Revision history for this message
weiyu (weiyu) wrote : Re:[Bug 1540234] Re: rabbitmq cluster is not work
Download full text (3.1 KiB)

you can set three hosts for rabbitmq clusterer.
------------------ Original ------------------
From: "Sam Yaple"<email address hidden>
Date: Thu, Feb 4, 2016 07:54 PM
To: "weiyu"<email address hidden>;
Subject: [Bug 1540234] Re: rabbitmq cluster is not work

This patch needs to be reverted as it breaks clusterer on ubuntu. Also,
the behavior you are describing I cannot reproduce and I have a - in my
hostname. The initial config is correct and clusterer should not be
quoted like this, there is another issue at play

====================

Working on ubuntu:

[
  {version, 1},
  {nodes, [
    {rabbit@y-work-laptop01, disc}
  ]},
  {gospel, {node, rabbit@y-work-laptop01}}
].

====================

Broken on ubuntu:

[
  {version, 1},
  {nodes, [
    {'rabbit@y-work-laptop01', disc}
  ]},
  {gospel, {node, 'rabbit@y-work-laptop01'}}
].

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1540234

Title:
  rabbitmq cluster is not work

Status in kolla:
  Confirmed

Bug description:
  rabbitmq cluster is not work

  bash-4.2$ rabbitmqctl cluster_status
  Cluster status of node 'rabbit@openstack-2' ...
  [{nodes,[{disc,['rabbit@openstack-2']}]},
   {running_nodes,['rabbit@openstack-2']},
   {cluster_name,<<"rabbit@openstack-2">>},
   {partitions,[]}]

  bash-4.2$ cat /etc/rabbitmq/rabbitmq_clusterer.config
  [
    {version, 1},
    {nodes, [
      {rabbit@openstack-1, disc},
      {rabbit@openstack-2, disc},
      {rabbit@openstack-3, disc}
    ]},
    {gospel, {node, rabbit@openstack-1}}
  ].

  bash-4.2$ rpm -q rabbitmq-server
  rabbitmq-server-3.5.5-3.noarch
  bash-4.2 rabbitmq-plugins list
   Configured: E = explicitly enabled; e = implicitly enabled
   | Status: * = running on rabbit@openstack-1
   |/
  [e*] amqp_client 3.5.5
  [ ] cowboy 0.5.0-rmq3.5.5-git4b93c2d
  [e*] mochiweb 2.7.0-rmq3.5.5-git680dba8
  [ ] rabbitmq_amqp1_0 3.5.5
  [ ] rabbitmq_auth_backend_ldap 3.5.5
  [ ] rabbitmq_auth_mechanism_ssl 3.5.5
  [E*] rabbitmq_clusterer 3.5.x-189b3a81
  [ ] rabbitmq_consistent_hash_exchange 3.5.5
  [ ] rabbitmq_federation 3.5.5
  [ ] rabbitmq_federation_management 3.5.5
  [E*] rabbitmq_management 3.5.5
  [e*] rabbitmq_management_agent 3.5.5
  [ ] rabbitmq_management_visualiser 3.5.5
  [ ] rabbitmq_mqtt 3.5.5
  [ ] rabbitmq_shovel 3.5.5
  [ ] rabbitmq_shovel_management 3.5.5
  [ ] rabbitmq_stomp 3.5.5
  [ ] rabbitmq_test 3.5.5
  [ ] rabbitmq_tracing 3.5.5
  [e*] rabbitmq_web_dispatch 3.5.5
  [ ] rabbitmq_web_stomp 3.5.5
  [ ] rabbitmq_web_stomp_examples 3.5.5
  [ ] sockjs 0.3.4-rmq3.5.5-git3132eb9
  [e*] webmachine 1.10.3-rmq3.5.5-gite9359c7

To manage notifications about this bug go to:
https://bugs.launchpad.net/kolla/+bug...

Read more...

Revision history for this message
Sam Yaple (s8m) wrote : Re: rabbitmq cluster is not work

weiyu I will attempt three hosts after mid-cycle on ubutnu

Steven Dake (sdake)
Changed in kolla:
milestone: none → mitaka-rc1
importance: Undecided → High
Steven Dake (sdake)
Changed in kolla:
milestone: mitaka-rc1 → mitaka-rc2
Revision history for this message
Steven Dake (sdake) wrote :

Sam,

Any progress on trying the 3 hosts on ubuntu? I don't have access to 3 hosts atm.

Steven Dake (sdake)
Changed in kolla:
importance: High → Critical
tags: added: rc-backport-potential
Steven Dake (sdake)
Changed in kolla:
milestone: mitaka-rc2 → newton-1
Steven Dake (sdake)
summary: - rabbitmq cluster is not work
+ rabbitmq cluster does not work with dashes in the hostname
tags: removed: rc-backport-potential
Revision history for this message
weiyu (weiyu) wrote :
Revision history for this message
Hui Kang (huikang27) wrote :

Is there anyone working on this bug?

Revision history for this message
Steven Dake (sdake) wrote :

weiyu,

try escaping erlang special characters (however that is done, perhaps \?) rather then quoting the full address.

Revision history for this message
Steven Dake (sdake) wrote :

Hui,
I'm not clear if weiyu is still working on this bug, but we need clusterer working on centos and ubuntu for both hosts with dashes and without. hosts with dashes in their names are a common pattern in the operational world.

Regards,
-steve

weiyu (weiyu)
Changed in kolla:
assignee: weiyu (weiyu) → nobody
Changed in kolla:
assignee: nobody → Michał Jastrzębski (inc007)
Changed in kolla:
status: Confirmed → In Progress
Revision history for this message
Steven Dake (sdake) wrote :

weiyu.

Thanks for fixing this bug. we have unreverted the revert and given you credit in the latest patch as a co-author as well as for the fix for 2.0.0. Apologies for the confusion.

Regards
-steve

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/288415
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=7829aa27b316ea69cb782ac89d705a3a731ba247
Submitter: Jenkins
Branch: master

commit 7829aa27b316ea69cb782ac89d705a3a731ba247
Author: Proskurin Kirill <email address hidden>
Date: Fri Mar 4 14:46:01 2016 +0300

    Add quotes for RBMQ nodes definition

    erlang parser, cant parse hostname with minus symbol, it returns:
    Ignoring external configuration due to error: {1,erl_parse,"bad term"}

    Adding single quotes fix this issue.

    Co-Authored-By: weiyu <email address hidden>
    Closes-Bug: #1540234
    Change-Id: I80e0789aa31febd552a851e6dc3a835d89c0e9d1

Changed in kolla:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/mitaka)

Reviewed: https://review.openstack.org/303919
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=55406395fc432507b89336a03f2d2f963e45120f
Submitter: Jenkins
Branch: stable/mitaka

commit 55406395fc432507b89336a03f2d2f963e45120f
Author: Proskurin Kirill <email address hidden>
Date: Fri Mar 4 14:46:01 2016 +0300

    Add quotes for RBMQ nodes definition

    erlang parser, cant parse hostname with minus symbol, it returns:
    Ignoring external configuration due to error: {1,erl_parse,"bad term"}

    Adding single quotes fix this issue.

    Co-Authored-By: weiyu <email address hidden>
    Closes-Bug: #1540234
    Change-Id: I80e0789aa31febd552a851e6dc3a835d89c0e9d1
    (cherry picked from commit 7829aa27b316ea69cb782ac89d705a3a731ba247)

Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote : Fix included in openstack/kolla 2.0.0.0rc4

This issue was fixed in the openstack/kolla 2.0.0.0rc4 release candidate.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/kolla 2.0.0

This issue was fixed in the openstack/kolla 2.0.0 release.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/kolla 1.1.0

This issue was fixed in the openstack/kolla 1.1.0 release.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/kolla 3.0.0.0b1

This issue was fixed in the openstack/kolla 3.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.