rabbitmqctl list commands are not resilient to network delays

Bug #1566265 reported by Alexey Lebedeff
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
Medium
Alexey Lebedeff
10.0.x
Won't Fix
Medium
Alexey Lebedeff
9.x
Won't Fix
Medium
Alexey Lebedeff

Bug Description

Detailed bug description:
   list_channels and list_queues are used by OCF script for monitoring, but this commands can fail in not-so-perfect network conditions. There is an upstream improvement that removes rpc roundtrip and makes this commands mostly ignore network delay - https://github.com/rabbitmq/rabbitmq-server/pull/683

Steps to reproduce:
  Introduce some small network delay using `tc` (e.g. 1ms) and look at how time it takes to list significant amount of queues skyrocket.

Expected results:
  Listing big number of queues/channels should always take approximately some amount of time, irregarding of network delay - examples of acceptable and unacceptable values are at https://github.com/rabbitmq/rabbitmq-server/pull/683#issuecomment-202325840

Actual result:
  See comparsion of old and new implementations - https://github.com/rabbitmq/rabbitmq-server/pull/683#issuecomment-202325840

Reproducibility:
 Always

Workaround:
 Use faster/less congested network.

Impact:
 False positives in OCF script could result in unnecessary rabbitmq restarts.

Changed in mos:
status: New → In Progress
importance: Undecided → High
importance: High → Medium
assignee: nobody → Alexey Lebedeff (alebedev-a)
milestone: none → 9.0
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/trusty/rabbitmq-server (master)

Fix proposed to branch: master
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/19186

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/trusty/rabbitmq-server (9.0)

Fix proposed to branch: 9.0
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/19190

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/centos7/rabbitmq-server (master)

Fix proposed to branch: master
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/19191

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/centos7/rabbitmq-server (9.0)

Fix proposed to branch: 9.0
Change author: Alexey Lebedeff <email address hidden>
Review: https://review.fuel-infra.org/19193

Changed in mos:
status: In Progress → Won't Fix
Revision history for this message
Dina Belova (dbelova) wrote :

Added move-to-10.0 tag due to the fact bug was transferred from 9.0 to 10.0

tags: added: move-to-10.0
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on packages/trusty/rabbitmq-server (master)

Change abandoned by Alexey Lebedeff <email address hidden> on branch: master
Review: https://review.fuel-infra.org/19186

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on packages/trusty/rabbitmq-server (9.0)

Change abandoned by Alexey Lebedeff <email address hidden> on branch: 9.0
Review: https://review.fuel-infra.org/19190

Changed in mos:
status: Won't Fix → Confirmed
tags: added: 10.0-reviewed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on packages/centos7/rabbitmq-server (master)

Change abandoned by Alexey Lebedeff <email address hidden> on branch: master
Review: https://review.fuel-infra.org/19191

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on packages/centos7/rabbitmq-server (9.0)

Change abandoned by Alexey Lebedeff <email address hidden> on branch: 9.0
Review: https://review.fuel-infra.org/19193

Revision history for this message
Alexey Lebedeff (alebedev-a) wrote :

Will be fixed by https://bugs.launchpad.net/fuel/+bug/1614071
`list_XXX` commands will remain unfixed until 3.7.0, but with the fix from above it no longer matters.

Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :

We removed list_XXX calls from RabbitMQ's OCF script in 9.1, hence the bug became irrelevant.

Changed in mos:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.