Comment 15 for bug 1835615

Reviewed: https://review.opendev.org/677404
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=914d66ad81dac2be83ff94ee54291fdbf5f7e3c7
Submitter: Zuul
Branch: stable/stein

commit 914d66ad81dac2be83ff94ee54291fdbf5f7e3c7
Author: Michele Baldessari <email address hidden>
Date: Wed Aug 14 16:40:46 2019 +0200

    Make the rabbitmq-ready exec more stringent

    Currently we use the following command to determine if rabbit is
    up and running *and* ready to service requests:
    rabbitmqctl eval "rabbit_mnesia:is_clustered()." | grep -q true

    Now we have occasionally observed that rabbitmqctl policies commands
    which are executed after said exec['rabbitmq-ready'] will fail.

    One potential reason is that is_clustered() can return true *before*
    the rabbit app is actually running. In fact we can see it does
    return true even though the app is stopped:
    ()[root@controller-1 /]$ rabbitmqctl stop_app
    Stopping rabbit application on node rabbit@controller-1 ...
    ()[root@controller-1 /]$ rabbitmqctl eval 'rabbit_mnesia:is_clustered().'
    true

    Let's switch to a combination of commands that check for the cluster to
    be up *and* the rabbitmq app to be running:
    ()[root@controller-1 /]$ rabbitmqctl stop_app
    Stopping rabbit application on node rabbit@controller-1 ...
    ()[root@controller-1 /]$ rabbitmqctl eval 'rabbit_nodes:is_running(node(), rabbit).'
    false

    Suggested-By: Bogdan Dobrelya <email address hidden>
    Closes-Bug: #1835615

    Change-Id: I29f779145a39cd16374a91626f7fae1581a18224