Hanging epmd leads to unrecoverable RabbitMQ failure

[Dmitry Mescheryakov]

Sometimes during normal operations RabbitMQ can not be restarted on a node. The following line could be found in /var/log/rabbitmq/startup_log:
node with name "rabbit" already running on "node-1

The only way to resolve the issue is to manually kill the epmd process. Then it will be respawned on the next RabbitMQ start and both it and RabbitMQ will operate normally.

Previously we added a clarification to the doc on how to resolve the situation if it happens in scope of bug https://bugs.launchpad.net/fuel/+bug/1479422 . Now we want to fully fix the issue.

[Davanum Srinivas (DIMS) (dims-v)]

Any updates on this Alexey?

[Alexey Lebedeff (alebedev-a)]

Yes, I believe I've found a solution to this bug. PR to upstream: https://github.com/erlang/otp/pull/923
Now waiting for feedback.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 8.0
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/16299

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/16299
Submitter: Pkgs Jenkins <email address hidden>
Branch: 8.0

Commit: 39b3c7f86e26ff48a0fb045e61fe2d96d8707974
Author: Alexey Lebedeff <email address hidden>
Date: Thu Jan 21 06:38:38 2016

Backport epmd fix

https://github.com/erlang/otp/pull/923 - prevent dead node from being
stuck in epmd

Change-Id: I7323bf621cd60529753202533c80b321be4c596a
Closes-Bug: #1525919

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: master
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/16473

[Artem Silenkov (asilenkov)]

Please assign to me when you finish this activity. It seems that we need this patch for centos too.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/16473
Submitter: Pkgs Jenkins <email address hidden>
Branch: master

Commit: 809547dae09c4a66d0abe8b548406ed6b1bbdb3d
Author: Alexey Lebedeff <email address hidden>
Date: Mon Feb 15 15:52:04 2016

Backport epmd fix

Cherry-pick 39b3c7f86e26ff48a0fb045e61fe2d96d8707974 from '8.0'

https://github.com/erlang/otp/pull/923 - prevent dead node from being
stuck in epmd.

Change-Id: I7323bf621cd60529753202533c80b321be4c596a
Closes-Bug: #1525919

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/18359

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18359
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: e61fde43d5024460783eb6d7314642867f9f2243
Author: Alexey Lebedeff <email address hidden>
Date: Tue Mar 22 17:41:07 2016

Backport epmd fix

Cherry-pick 39b3c7f86e26ff48a0fb045e61fe2d96d8707974 from '8.0'

https://github.com/erlang/otp/pull/923 - prevent dead node from being
stuck in epmd.

Change-Id: I7323bf621cd60529753202533c80b321be4c596a
Closes-Bug: #1525919
(cherry picked from commit 809547dae09c4a66d0abe8b548406ed6b1bbdb3d)

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

version

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Dmitry Mescheryakov (dmitrymex)]

Fix https://review.fuel-infra.org/18359 is merged.

[Alexey Galkin (agalkin)]

Verified as fixed in 9.0-220