Could not contact node / connection attempt from disallowed node on new installation
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack RabbitMQ Server Charm |
Expired
|
Undecided
|
Unassigned | ||
Rabbitmq Plugins Common |
Invalid
|
Undecided
|
Unassigned |
Bug Description
Using charm cs:rabbitmq-server revision 75 on xenial/queens cloud with only 1 rabbitmq-server node in the cluster, deployment of rabbitmq-server with all-default options gives an error.
After the message 'Enabling queue mirroring', the charm hangs with a message 'hook failed: "config-changed"'.
The Juju unit-log (find full log attached) indicates applying the plugin configuration fails:
2018-07-16 08:19:30 DEBUG config-changed Applying plugin configuration to rabbit@
2018-07-16 08:19:30 DEBUG config-changed * Could not contact node rabbit@
2018-07-16 08:19:30 DEBUG config-changed Changes will take effect at broker restart.
2018-07-16 08:19:30 DEBUG config-changed * Options: --online - fail if broker cannot be contacted.
2018-07-16 08:19:30 DEBUG config-changed --offline - do not try to contact broker.
The rabbit access logs indicated it finds the connection attempt disallowed:
=ERROR REPORT==== 16-Jul-
** Connection attempt from disallowed node 'rabbitmq-
RabbitMQs startup_log (startup_err is empty):
## ## Licensed under the MPL. See http://
## ##
########## Logs: /<email address hidden>
###### ## /<email address hidden>
##########
Forward and Reverse DNS (both FQDN and search) give correct results and the hosts-file also contains the right hostname to IP address entry.
I suspect this may be related to the "bit racey" behaviour described in the README, however, running `juju resolved rabbitmq-server/0` does not solve the problem. What worked on 2 occassions is to `juju debug rabbitmq-server/0`, run `juju resolved rabbitmq-server/0` from another session, wait for a couple of minutes, exit the debug-session and set to resolved again and suddenly it picks up again. The charm rerunning the config-changed hook periodically, however, does not solve the problem.
Afterwards, the startup_log indicates the broker starts with 6 plugins and openstack installation finishes succesful.
Additionally, last time the debug-hooks trick didn't work until I restarted the rabbitmq-server using `service rabbitmq-server restart`.