Failed to find required executable rabbitmqctl

[Satish Patel]

I am building multi node cluster with 3 infra node (ostack-controller-X) and 1 compute node with CentOS7 64bit, now i am stuck here.

[root@ostack-osa playbooks]# openstack-ansible setup-openstack.yml
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml "

PLAY [Prepare MQ/DB services] *****************************************************************************************************************************************************

TASK [Ensure Rabbitmq vhost] ******************************************************************************************************************************************************
Tuesday 22 May 2018 13:22:56 -0400 (0:00:00.104) 0:00:00.105 ***********
fatal: [ostack-controller-03_keystone_container-503bf55e -> 192.168.100.189]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to find required executable rabbitmqctl in paths: /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"}

NO MORE HOSTS LEFT ****************************************************************************************************************************************************************

PLAY RECAP ************************************************************************************************************************************************************************
ostack-controller-03_keystone_container-503bf55e : ok=0 changed=0 unreachable=0 failed=1

Tuesday 22 May 2018 13:23:00 -0400 (0:00:04.210) 0:00:04.315 ***********
===============================================================================
Ensure Rabbitmq vhost --------------------------------------------------- 4.21s

EXIT NOTICE [Playbook execution failure] **************************************
===============================================================================

[Satish Patel (satish-txt)]

after running -vvv i saw following errors

container_name: "ostack-controller-03_rabbit_mq_container-c482ee13"
physical_host: "ostack-controller-03"
Container confirmed
<192.168.100.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.100.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=5 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=64 -o ServerAliveCountMax=1024 -o Compression=no -o TCPKeepAlive=yes -o VerifyHostKeyDNS=no -o ForwardX11=no -o ForwardAgent=yes -T -o ControlPath=/root/.ansible/cp/cf55a70db6 -tt 192.168.100.3 'lxc-attach --clear-env --name ostack-controller-03_rabbit_mq_container-c482ee13 -- su - root -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/usr/bin/python /root/.ansible/tmp/ansible-tmp-1527013772.43-257309099288749/rabbitmq_vhost.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1527013772.43-257309099288749/" > /dev/null 2>&1 && sleep 0'"'"'"'"'"'"'"'"''"'"''
<192.168.100.3> (0, '\r\n{"msg": "Failed to find required executable rabbitmqctl in paths: /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin", "failed": true, "invocation": {"module_args": {"node": "rabbit", "tracing": false, "state": "present", "name": "/keystone"}}}\r\n', 'Shared connection to 192.168.100.3 closed.\r\n')
fatal: [ostack-controller-03_keystone_container-503bf55e -> 192.168.100.189]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "name": "/keystone",
            "node": "rabbit",
            "state": "present",
            "tracing": false
        }
    },
    "msg": "Failed to find required executable rabbitmqctl in paths: /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"
}

NO MORE HOSTS LEFT ****************************************************************************************************************************************************************

PLAY RECAP ************************************************************************************************************************************************************************
ostack-controller-03_keystone_container-503bf55e : ok=0 changed=0 unreachable=0 failed=1

Tuesday 22 May 2018 14:29:34 -0400 (0:00:04.134) 0:00:04.241 ***********
===============================================================================
Ensure Rabbitmq vhost --------------------------------------------------- 4.14s
/var/opt/openstack-ansible/playbooks/common-tasks/rabbitmq-vhost-user.yml:21 --

EXIT NOTICE [Playbook execution failure] **************************************
===============================================================================

[Satish Patel (satish-txt)]

I found my openstack-infra was failed which cause that error and i re-run openstack-infra and notice this error.

http://paste.openstack.org/show/721655/

[Jean-Philippe Evrard (jean-philippe-evrard)]

the setup-openstack need the setup-infrastructure to be well deployed beforehand.
Here you had an issue on setup-infrastructure, so that's the reason of the failure.

The question now is why the setup-infrastructure failed.
I suggest you re-run the setup-hosts.yml and setup-infrastructure.yml, and paste their logs somewhere so we can help you further.

In the meantime, I think this bug is invalid.

[Jean-Philippe Evrard (jean-philippe-evrard)]

Next to that, if you paste the output here, could you also add the output of the 'getenforce' command, please?

Thank you in advance.