Message queue for SUSE in Installation Guide

[Buddhika Sanjeewa]

This bug tracker is for errors with the documentation, use the following as a template and remove or add fields as you see fit. Convert [ ] into [x] to check boxes:

- [ ] This doc is inaccurate in this way: ______
- [ ] This is a doc addition request.
- [x] I have a fix to the document that I can paste below including example:

OS: SLES 12 SP 3 (IPv6 disabled)
OpenStack version : Pike

When I run "systemctl start rabbitmq-server.service" it always fails with the following log in /var/log/messages

=============================================
2018-01-11T15:35:34.480503+05:30 controller systemd[1]: Reloading.
2018-01-11T15:35:34.610525+05:30 controller systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
2018-01-11T15:35:34.644170+05:30 controller systemd[1]: Reloading.
2018-01-11T15:35:34.773331+05:30 controller systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
2018-01-11T15:35:40.803131+05:30 controller systemd[1]: Reloading.
2018-01-11T15:35:40.932390+05:30 controller systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
2018-01-11T15:35:48.045112+05:30 controller systemd[1]: Listening on Erlang Port Mapper Daemon Activation Socket.
2018-01-11T15:35:48.059994+05:30 controller systemd[1]: Started Erlang Port Mapper Daemon.
2018-01-11T15:35:48.062549+05:30 controller systemd[1]: Starting RabbitMQ broker...
2018-01-11T15:35:48.591676+05:30 controller systemd[1]: rabbitmq-server.service: Main process exited, code=exited, status=1/FAILURE
2018-01-11T15:35:49.659212+05:30 controller rabbitmqctl[8780]: Stopping and halting node rabbit@controller
2018-01-11T15:35:49.667661+05:30 controller rabbitmqctl[8780]: Error: unable to connect to node rabbit@controller: nodedown
2018-01-11T15:35:49.679173+05:30 controller rabbitmqctl[8780]: DIAGNOSTICS
2018-01-11T15:35:49.679771+05:30 controller rabbitmqctl[8780]: ===========
2018-01-11T15:35:49.680173+05:30 controller rabbitmqctl[8780]: attempted to contact: [rabbit@controller]
2018-01-11T15:35:49.680595+05:30 controller rabbitmqctl[8780]: rabbit@controller:
2018-01-11T15:35:49.681037+05:30 controller rabbitmqctl[8780]: * unable to connect to epmd (port 4369) on controller: address (cannot connect to host/port)
2018-01-11T15:35:49.681415+05:30 controller rabbitmqctl[8780]: current node details:
2018-01-11T15:35:49.681705+05:30 controller rabbitmqctl[8780]: - node name: 'rabbitmq-cli-54@controller'
2018-01-11T15:35:49.681988+05:30 controller rabbitmqctl[8780]: - home dir: /var/lib/rabbitmq
2018-01-11T15:35:49.682270+05:30 controller rabbitmqctl[8780]: - cookie hash: R6aclPnr+k4dau9mL3XKrg==
2018-01-11T15:35:49.686279+05:30 controller systemd[1]: Failed to start RabbitMQ broker.
2018-01-11T15:35:49.686686+05:30 controller systemd[1]: rabbitmq-server.service: Unit entered failed state.
2018-01-11T15:35:49.687026+05:30 controller systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.
2018-01-11T15:44:31.709338+05:30 controller systemd[1]: Stopping Erlang Port Mapper Daemon...
2018-01-11T15:44:31.711309+05:30 controller systemd[1]: Stopped Erlang Port Mapper Daemon.

=======================================================================

I referred to https://docs.openstack.org/liberty/install-guide-obs/environment-messaging.html and did the following.

Copy the /usr/lib/systemd/system/epmd.socket file to the /etc/systemd/system directory.

Edit the /etc/systemd/system/epmd.socket file to contain the following:

    [Socket]
    ...
    ListenStream=127.0.0.1:4369
    ListenStream=MANAGEMENT_INTERFACE_IP_ADDRESS:4369

Then "systemctl start rabbitmq-server.service" works.

-----------------------------------
Release: on 2018-01-09 21:23
SHA: f937e6cd146be5b3e2ff1b07de4c2f7171d23b98
Source: https://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/source/environment-messaging-obs.rst
URL: https://docs.openstack.org/install-guide/environment-messaging-obs.html

[Nicolas Bock (nicolasbock)]

I can't reproduce this problem on SLES-12-SP3. Did you add all necessary entries to your `/etc/hosts` file? I found that if those entries are not correct, then rabbitmq refuses to start.

[Buddhika Sanjeewa (bsanjeewa)]

The /etc/hosts file contains controller and localhost entries, and they are correct.
Had done the installation twice after reformatting. Could only start rabbitmq after editing the /etc/systemd/system/epmd.socket file. First time set the line as
    ListenStream=:::4369
Then it worked. But wanted to specify the IP so the second time edited as in the first post.

ListenStream=127.0.0.1:4369
ListenStream=MANAGEMENT_INTERFACE_IP_ADDRESS:4369

[Buddhika Sanjeewa (bsanjeewa)]

I'll try changing the above file to default and retry on Monday.

[Buddhika Sanjeewa (bsanjeewa)]

Test 1:
With /etc/systemd/system/epmd.socket as
-----------------------------------
[Unit]
Description=Erlang Port Mapper Daemon Activation Socket

[Socket]
ListenStream=127.0.0.1:4369
#ListenStream=10.101.0.11:4369
Accept=false

[Install]
WantedBy=sockets.target
-------------------------------------

controller:~ # service rabbitmq-server status
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2018-01-15 09:42:46 +0530; 48s ago
  Process: 1794 ExecStop=/usr/sbin/rabbitmqctl stop (code=exited, status=0/SUCCESS)
  Process: 1478 ExecStart=/usr/sbin/rabbitmq-server (code=exited, status=1/FAILURE)
 Main PID: 1478 (code=exited, status=1/FAILURE)

Jan 15 09:42:46 controller rabbitmqctl[1794]: attempted to contact: [rabbit@controller]
Jan 15 09:42:46 controller rabbitmqctl[1794]: rabbit@controller:
Jan 15 09:42:46 controller rabbitmqctl[1794]: * unable to connect to epmd (port 4369) on controller: address (cannot connect to host/port)
Jan 15 09:42:46 controller rabbitmqctl[1794]: current node details:
Jan 15 09:42:46 controller rabbitmqctl[1794]: - node name: 'rabbitmq-cli-79@controller'
Jan 15 09:42:46 controller rabbitmqctl[1794]: - home dir: /var/lib/rabbitmq
Jan 15 09:42:46 controller rabbitmqctl[1794]: - cookie hash: R6aclPnr+k4dau9mL3XKrg==
Jan 15 09:42:46 controller systemd[1]: Failed to start RabbitMQ broker.
Jan 15 09:42:46 controller systemd[1]: rabbitmq-server.service: Unit entered failed state.
Jan 15 09:42:46 controller systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.

Test 2:

With /etc/systemd/system/epmd.socket as
-----------------------------------
[Unit]
Description=Erlang Port Mapper Daemon Activation Socket

[Socket]
#ListenStream=127.0.0.1:4369
ListenStream=10.101.0.11:4369
Accept=false

[Install]
WantedBy=sockets.target
-----------------------------------

controller:~ # service rabbitmq-server status
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
   Active: inactive (dead)

Jan 15 09:46:22 controller systemd[1]: Dependency failed for RabbitMQ broker.
Jan 15 09:46:22 controller systemd[1]: rabbitmq-server.service: Job rabbitmq-server.service/start failed with result 'dependency'.

controller:~ # service rabbitmq-server start
Job for rabbitmq-server.service failed because the control process exited with error code. See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.

Test 3:
With /etc/systemd/system/epmd.socket as
-------------------------------------
[Unit]
Description=Erlang Port Mapper Daemon Activation Socket

[Socket]
ListenStream=127.0.0.1:4369
ListenStream=10.101.0.11:4369
Accept=false

[Install]
WantedBy=sockets.target
-------------------------------------
controller:~ # service rabbitmq-server status
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
   Active: inactive (dead)

Jan 15 09:57:38 controller systemd[1]: Dependency failed...

[Nicolas Bock (nicolasbock)]

I found that:

cat /etc/systemd/system/epmd.socket.d/override.conf
[Socket]
ListenStream=10.0.0.11:4369

works as well. Could you verify that this does the trick in your setup?

[Buddhika Sanjeewa (bsanjeewa)]

Yes this works. Service starts as expected.
On my configuration, /etc/systemd/system/epmd.socket.d/ was not available, so I had to create it.

Now on my system,
-----------------
cat /etc/systemd/system/epmd.socket
[Unit]
Description=Erlang Port Mapper Daemon Activation Socket

[Socket]
ListenStream=127.0.0.1:4369
Accept=false

[Install]
WantedBy=sockets.target
---------------------
and
--------------------
cat /etc/systemd/system/epmd.socket.d/override.conf
[Socket]
ListenStream=MANAGEMENT_IP:4369
--------------------

[Buddhika Sanjeewa (bsanjeewa)]

But the service does not start on system startup. Any advice?