Massive memory usage in rabbit node with trusty-liberty cloud deployment

[Billy Olsen]

RabbitMQ server on primary node in a rabbitmq-server cluster uses all available memory over time for the server. This causes client connection to become blocked due to low memory available on the rabbitmq-server.

The rabbitmq stats collection was consuming the majority of the database. This charm provides no option to tune stats collection and should.

This is documented in the OpenStack Ops Guide at http://docs.openstack.org/ops-guide/ops-maintenance-rabbitmq.html under the heading "Excessive database management memory consumption"

[David Ames (thedac)]

"Edit the file:etc/rabbitmq/rabbitmq.config configuration file, and change the collect_statistics_interval parameter between 30000-60000 milliseconds. Alternatively you can turn off statistics collection by setting collect_statistics parameter to “none”."

Need to add configuration values for both collect_statistics_interval and collect_statistics so this is configurable.

rabbitmq.config will look something like this:
[
    {rabbit, [

        {tcp_listeners, [5672]},
        {collect_statistics_interval, 30000},

        {cluster_partition_handling, autoheal}

    ]}
].

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/431128

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/431128
Committed: https://git.openstack.org/cgit/openstack/charm-rabbitmq-server/commit/?id=4978fe304b5c843741be0d2dd69ff04ebc4fa611
Submitter: Jenkins
Branch: master

commit 4978fe304b5c843741be0d2dd69ff04ebc4fa611
Author: Shane Peters <email address hidden>
Date: Wed Feb 8 14:40:41 2017 -0500

    Change default collect_statistics_interval to 30s

    By default, the management plugin collects statistics every 5 seconds.
    This modifies it to 30s in order to alleviate memory pressure. This
    value is only considered when the management plugin is enabled.

    Change-Id: I7dbbcfc1cbc0a17c6a3c421914883e52cf74afb5
    Closes-Bug: 1656074

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/16.10
Review: https://review.openstack.org/432348

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/432348
Committed: https://git.openstack.org/cgit/openstack/charm-rabbitmq-server/commit/?id=ddeed234ef6dee87f66bdc439d9616dd9b917d2f
Submitter: Jenkins
Branch: stable/16.10

commit ddeed234ef6dee87f66bdc439d9616dd9b917d2f
Author: Shane Peters <email address hidden>
Date: Wed Feb 8 14:40:41 2017 -0500

    Change default collect_statistics_interval to 30s

    By default, the management plugin collects statistics every 5 seconds.
    This modifies it to 30s in order to alleviate memory pressure. This
    value is only considered when the management plugin is enabled.

    Change-Id: I7dbbcfc1cbc0a17c6a3c421914883e52cf74afb5
    Closes-Bug: 1656074