RabbitMQ doesn't start after master node reset

Bug #1319951 reported by Mike Scherbakov
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
High
Matthew Mosesohn

Bug Description

After hard reset of master node, RabbitMQ looks started but doesn't work properly:

[root@d075c6249142 rabbitmq]# rabbitmqctl list_queues
Listing queues ...
Error: unable to connect to node rabbit@fuel: nodedown

DIAGNOSTICS
===========

nodes in question: [rabbit@fuel]

hosts, their running nodes and ports:
- unable to connect to epmd on fuel: nxdomain (non-existing domain)

current node details:
- node name: rabbitmqctl1149@d075c6249142
- home dir: /var/lib/rabbitmq
- cookie hash: 2xPc5aEDV5CIGSlYHzj7Nw==

[root@d075c6249142 rabbitmq]# hostname
d075c6249142
[root@d075c6249142 rabbitmq]# exit
[root@fuel ~]# hostname
fuel.domain.tld

If I try to restart it:
[root@d075c6249142 ~]# /etc/init.d/rabbitmq-server restart
Restarting rabbitmq-server: RabbitMQ is not running
RabbitMQ server startup failed on retry 1 of 60!
RabbitMQ server startup failed on retry 2 of 60!
RabbitMQ server startup failed on retry 3 of 60!
RabbitMQ server startup failed on retry 4 of 60!
RabbitMQ server startup failed on retry 5 of 60!
RabbitMQ server startup failed on retry 6 of 60!
RabbitMQ server startup failed on retry 7 of 60!
....
[root@d075c6249142 ~]# cat /var/log/rabbitmq/startup_log
ERROR: epmd error for host "fuel": nxdomain (non-existing domain) [FAILED]

I guess it can be related to hostname.

{"build_id": "2014-05-13_12-25-26", "mirantis": "yes", "build_number": "199", "ostf_sha": "bff980b2277a5898266500a26224a03bad594612", "nailgun_sha": "192b32e5ebd78b8521ced8329c07665dd75f5948", "production": "docker", "api": "1.0", "fuelmain_sha": "ac5500c45b1b05a72caadc0c7ef36c4279e4b36d", "astute_sha": "1b97b4d8832c12a1000235175b960e0102c23ca5", "release": "5.0", "fuellib_sha": "027751fc9c9806c9eb38683baf8bc1faced85de4"}

Mike Scherbakov (mihgen)
description: updated
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

it is 100% related to the hostname. you should use hostname as -n argument value when running rabbitmqctl and you should not use init script as it is not how rabbitmq should be restarted.

Changed in fuel:
status: New → Confirmed
assignee: Fuel Library Team (fuel-library) → Matthew Mosesohn (raytrac3r)
tags: added: ha
tags: removed: ha
Revision history for this message
Mike Scherbakov (mihgen) wrote :

How should I restart rabbitmq? It is master node BTW, not OpenStack nodes.

Changed in fuel:
status: Confirmed → Triaged
importance: Undecided → High
Revision history for this message
Mike Scherbakov (mihgen) wrote :

I can't reproduce an issue on #209..

Changed in fuel:
status: Triaged → Incomplete
Mike Scherbakov (mihgen)
Changed in fuel:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.