Add ability to disable HA for RabbitMQ queues

[OpenStack Infra]

https://review.openstack.org/249180
Dear bug triager. This bug was created since a commit was marked with DOCIMPACT.

commit 7e96ef47caeae03cf9696a67817e1756e1b63f4a
Author: Dmitry Mescheryakov <email address hidden>
Date: Wed Nov 18 13:30:46 2015 +0300

    Add ability to disable HA for RabbitMQ queues

    Add two flags:
     * enable_rpc_ha which enables queue mirroring for RPC queues
     * enable_notifications_ha which enables queue mirroring for
       Ceilometer queues

    Since the feature is experimental, both flags are set to true by
    default to preserve current behaviour.

    The change is implemented in several steps:
     * the upstream script changed so that it allows to extend the
       list of parameters and uses a policy file to define RabbitMQ
       policies.
     * we add our own version of OCF script which wraps around the
       upstream one. It defines a new enable_rpc_ha and
       enable_notifications_ha parameter and passes their value to the
       upstream script.
     * we add our policy file, where we use the introduced parameters
       to decide which policies we should set.

    So we will have two OCF scripts for RabbitMQ in our deployment:
     * rabbitmq-server-upstream - the upstream version
     * rabbitmq-server - our extention, which will be used in the
       environment

    The upstream version of the script is pushed to the upstream
    along with empty policy file, so that other users can define their
    own policies or extend the script if needed. Here are the
    corresponding pull requests:
      https://github.com/rabbitmq/rabbitmq-server/pull/480
      https://github.com/rabbitmq/rabbitmq-server/pull/482
    (both are already merged)

    Text for Operations Guide

    It is possible to significantly reduce load which OpenStack puts on
    RabbitMQ by disabling queue mirroring. This could be done separately
    for RPC queues and Ceilometer ones. To disable mirroring for RPC
    queues, execute the following command on one of the controllers:

        crm_resource --resource p_rabbitmq-server --set-parameter \
            enable_rpc_ha --parameter-value false

    To disable mirroring for Ceilometer queues, execute the following
    command on one of the controllers:

        crm_resource --resource p_rabbitmq-server --set-parameter \
            enable_notifications_ha --parameter-value false

    In order for any of the changes to take effect, RabbitMQ service
    should be restarted. To do that, first execute

        pcs resource disable master_p_rabbitmq-server

    Then monitor RabbitMQ state using command

        pcs resource

    until it shows that all RabbitMQ nodes are stopped. Once they are,
    execute the following command to start RabbitMQ:

        pcs resource enable master_p_rabbitmq-server

    Beware: during restart all messages accumulated in RabbitMQ will be
    lost. Also, OpenStack will stop functioning until RabbitMQ is up
    again, so plan accordingly.

    Note that it is not yet well tested how this configuration affects
    failover when some cluster nodes go down. Hence it is experimental,
    use at your own risk!

    DocImpact: ops-guide

    Implements: blueprint rabbitmq-disable-mirroring-for-rpc
    Change-Id: I80ae231ca64e2a903b0968d36ba0e85ca9cc9891

[Evgeny Konstantinov (evkonstantinov)]

in the knowledge base