rabbitmq_global_publishers and rabbitmq_global_consumers fail to handle unclean client termination

[Jorge Merlino]

[Impact]

When a client terminates uncleanly, the global counters for publishers and consumers do not handle this gracefully.

For the consumer metric, the value of the gauge does not decrease after the termination, and so the value ever-increases.
For the publisher metric, the value of the gauge decreases below zero and does not recover, leading to a ever-decreasing negative value on the gauge.

[Test plan]

lxc launch ubuntu:jammy rabbitmq-jammy-test

lxc shell rabbitmq-jammy-test

apt install -y rabbitmq-server

echo '[rabbitmq_prometheus].' > /etc/rabbitmq/enabled_plugins

systemctl restart rabbitmq-server.service

wget https://github.com/rabbitmq/rabbitmq-perf-test/releases/download/v2.21.0/perf-test-2.21.0.jar

apt install -y default-jre

for i in $(seq 1 5); do java -jar perf-test-2.21.0.jar --time 5; curl -s 127.0.0.1:15692/metrics | grep -E 'rabbitmq_global_(consumers|publishers){'; done

The output of this command shows the value of the rabbitmq_global_consumers and rabbitmq_global_publishers after each execution of the test. The problem described of monotonically increasing and decreasing values will be observed. As we are connecting and disconnecting a single client the expected value should be 0 for both counters.

[Where problems could occur]

This is a simple patch that calls the functions for deleting consumers and publishers in the correct places. It should not break anything else than the counters themselves but they were broken to begin with.

[Other Info]

This was patched upstream from version 3.10 so this patch is already applied to Noble and Oracular that use version 3.12. This is the upstream patch: https://github.com/rabbitmq/rabbitmq-server/pull/6244

[Jorge Merlino (jorge-merlino)]

Patch for Jammy

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "lp2073932_jammy.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Mauricio Faria de Oliveira (mfo)]

Hi Jorge,

Thanks for the debdiff!

It looks mostly good; I just noticed 3 points.

I'd adjust and upload myself weren't it for point 3, which I'm unsure about.
Could you please adjust and clarify?

1) In Origin:, please use the URL for the git commit in the repository, not in its PR, as they differ.
In this instance: PR/debdiff has f8a8fb7 and merged commit is d4c95b2 [1], linked from the PR.

2) In Origin:, use used 'upstream', but there are changes from the upstream patch (file/hunk removed), so this would be 'backport' per DEP-3 [2].

+ deps/rabbit/src/rabbit_channel.erl | 9 +++++++--
+ deps/rabbit/test/queue_type_SUITE.erl | 9 +++++++++
...
+diff --git a/deps/rabbit/src/rabbit_channel.erl b/deps/rabbit/src/rabbit_channel.erl
(no other +diff line for queue_type_SUITE.erl)

3) Why is the patch file/hunk for the tests (queue_type_SUITE.erl) dropped?
If this is needed for any reason, it should be noted in the backport notes, ideally, or in Other Info.

Thanks!
Mauricio

[1] https://github.com/rabbitmq/rabbitmq-server/commit/d4c95b2be33c0a2231cf60437af1eb6d221a3b2d
[2] https://dep-team.pages.debian.net/deps/dep3/

[Mauricio Faria de Oliveira (mfo)]

Marking the devel series (Oracular) as Fix Released, as 3.12 has the fix (verified).

Please also update that in future bugs :)