Nagios check plugin check_rabbitmq_queues doesn't output the busiest queues when specifying wildcards in argument

[Julien Thieffry]

This behavior has been observed in:
OS-Version: Ubuntu 18.04.5
Bootstack series: queens-bionic
Rabbitmq-server charm revision: 110

When wildcards for vhost and queue are specified in queue_thresholds (such as \* \* 100 200), whenever warnings or errors arise in the output of the check_rabbitmq_queues.py script (and Nagios dashboard), output is similar to: "CRITICAL: * in * has XXXX messages". While this properly reports a problem, information on which vhost/queues combination is being busy is lost as this value is being consolidated and reported under the wildcard.

This might be the expected behavior, but whenever such scenario happens, we often need to investigate which queues in particular are being busy. Since we use the default configuration option (wildcard on vhost and queue), current output from Nagios offers little help in identifying where the issue is coming from.

As a result, it would be helpful to specify a charm configuration parameter that allows to display an arbitrary number of the busiest queues in the script output whenever a warning or critical code is issued.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-rabbitmq-server/+/803669

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-rabbitmq-server/+/803669
Committed: https://opendev.org/openstack/charm-rabbitmq-server/commit/242167b6bad190dd7aa2b885272906551d390c8a
Submitter: "Zuul (22348)"
Branch: master

commit 242167b6bad190dd7aa2b885272906551d390c8a
Author: Julien Thieffry <email address hidden>
Date: Fri Aug 6 02:16:54 2021 +0000

    Display busiest queues in check_queues NRPE plugin

    When invoking the check_rabbitmq_queues script with wildcards for vhost
    and/or queue parameters, script output does not reflect precisely which
    queues are having a high number of oustanding messages as information is
    consolidated under the wildcard.

    This change fixes this behaviour by adding a new charm configuration
    parameter which allows the user to specify the number of busiest queues,
    n, to display should the check_rabbitmq_queues script reports any
    warnings or errors. The default, n=0, keeps the current script output.
    This option is applicable regardless of the vhost:queue combination but
    is specifically relevant when wildcards are passed as arguments.

    Implementation displays the first n items in the stats list re-organized
    in decreasing message count order.

    Closes-Bug: #1939084
    Change-Id: I5a32cb6bf37bd2a0f30861eace3c0e6cb5c2559d