rabbitmq deployment failure on mixed OS environment

Bug #1815301 reported by Kevin Carter
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Undecided
Unassigned

Bug Description

When deploying in an environment where an infra host may be ubuntu, suse, or centos the deployment of rabbitmq will fail due to the versions of rabbitmq being to different between the nodes.

```
TASK [rabbitmq_server : Join rabbitmq cluster] **************************************************************************************************************************************************************************************************************Saturday 09 February 2019 18:20:42 +0000 (0:00:06.115) 0:01:49.926 *****
FAILED - RETRYING: Join rabbitmq cluster (5 retries left).
FAILED - RETRYING: Join rabbitmq cluster (4 retries left).
FAILED - RETRYING: Join rabbitmq cluster (3 retries left).
FAILED - RETRYING: Join rabbitmq cluster (2 retries left).
FAILED - RETRYING: Join rabbitmq cluster (1 retries left).
fatal: [infra-centos]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["rabbitmqctl", "join_cluster", "rabbit@utility1-rabbit-mq-container-8b49b19c"], "delta": "0:00:01.054852", "end": "2019-02-09 18:20:58.841189", "msg": "non-zero return code", "rc": 2, "start": "2019-02-09 18:20:57.786337", "stderr": "Error: {inconsistent_cluster,\"Rabbit version mismatch: local node is 3.6.5, remote node 3.7.8\"}", "stderr_lines": ["Error: {inconsistent_cluster,\"Rabbit version mismatch: local node is 3.6.5, remote node 3.7.8\"}"], "stdout": "Clustering node 'rabbit@infra-centos' with 'rabbit@utility1-rabbit-mq-container-8b49b19c' ...", "stdout_lines": ["Clustering node 'rabbit@infra-centos' with 'rabbit@utility1-rabbit-mq-container-8b49b19c' ..."]}
```

It would seem that the capability to file install rabbitmq has been removed on centos and suse which makes it impossible to upgrade an environment with mixed operating systems. To correct the issue, rabbitmq needs to ONLY ever be installed on a single OS type or the deployer has to downgrade rabbitmq on the ubuntu host to meet that of the other distros.

Revision history for this message
Dmitriy Rabotyagov (noonedeadpunk) wrote :

We're defaulting to external_repo installation for a while, which works as long as repo contains requested package versions.

So this is not always reliable, since repos for rabbitmq tend to drop older packages from time to time.
We try to maintain minor upgrades once this happens, but it's not always possible.

General solution in case external_repo is missing would be either to find existing version and define with rabbitmq_package_version / rabbitmq_erlang_version_spec or use rabbitmq_install_method: distro.

Changed in openstack-ansible:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.