rabbitmq-server startup does not wait long enough

Bug #1870619 reported by Nicolas Bock
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack RabbitMQ Server Charm
Incomplete
Undecided
Nicolas Bock

Bug Description

[Impact]

 * Rabbitmq-server has 2 configuration settings that affect how long it will wait for the mnesia database to become available
 * The default is 30 seconds x 10 retries = 300 seconds
 * The startup wrapper rabbitmq-server-wait will wait only 10 seconds
 * If the database does not come online within 10 seconds the startup script will fail despite the fact that rabbitmq-server is still waiting for another 290 seconds.
 * This behavior leads to falsely identified failures in OpenStack for example when a Rabbitmq cluster is restarted out of order (LP: #1828988)

[Test Case]

 * Create Rabbitmq cluster and create a queue with "ha-mode: all" policy
 * Shut down nodes one by one
 * Restart the node that was shut down first
 * This node will fail to start because it was not the master of the queue
 * Note that the startup script (SysV or systemd) will fail after 10 seconds while the rabbitmq-server process is still waiting for the database to come online

[Regression Potential]

 * This change potentially increases the time the rabbitmq-server service takes to start up which might lead to failures down the dependency chain of startup services.
 * This change potentially changes the result of starting the rabbitmq-server service in case the mnesia database takes more than 10 seconds to come online. Without this change, the service will incorrectly fail while it will succeed with this change.

Tags: patch sts
Changed in charm-rabbitmq-server:
assignee: nobody → Nicolas Bock (nicolasbock)
Revision history for this message
Nicolas Bock (nicolasbock) wrote :
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "rabbitmq-server.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in rabbitmq-server (Ubuntu Bionic):
status: New → Confirmed
Changed in rabbitmq-server (Ubuntu Disco):
status: New → Confirmed
Changed in rabbitmq-server (Ubuntu Eoan):
status: New → Confirmed
Changed in rabbitmq-server (Ubuntu):
status: New → Confirmed
Eric Desrochers (slashd)
tags: added: sts
Dan Streetman (ddstreet)
Changed in rabbitmq-server (Ubuntu Disco):
status: Confirmed → Won't Fix
Changed in rabbitmq-server (Ubuntu Focal):
assignee: nobody → Nicolas Bock (nicolasbock)
Changed in rabbitmq-server (Ubuntu Eoan):
assignee: nobody → Nicolas Bock (nicolasbock)
Changed in rabbitmq-server (Ubuntu Bionic):
assignee: nobody → Nicolas Bock (nicolasbock)
importance: Undecided → Medium
Changed in rabbitmq-server (Ubuntu Eoan):
importance: Undecided → Medium
Changed in rabbitmq-server (Ubuntu Focal):
importance: Undecided → Medium
Revision history for this message
Eric Desrochers (slashd) wrote :

IIUC, this is your own patch (you being the author).

* Does upstream maintain the debian/ folder ?
 ** If yes, can you please make sure to submit a PR upstream ?
* Can you file a bug and submit the patch to Debian as well ?
 ** https://wiki.debian.org/BugReport

Thanks !

description: updated
no longer affects: rabbitmq-server (Ubuntu Disco)
Revision history for this message
Nicolas Bock (nicolasbock) wrote :

I created another bug LP: #1874075 that tracks changes to the rabbitmq-server package.

no longer affects: rabbitmq-server (Ubuntu)
no longer affects: rabbitmq-server (Ubuntu Bionic)
no longer affects: rabbitmq-server (Ubuntu Eoan)
no longer affects: rabbitmq-server (Ubuntu Focal)
Revision history for this message
Billy Olsen (billy-olsen) wrote :

I've marked this bug as incomplete against the charm as it appears that it is an issue with the rabbitmq-server package, being tracked in a separate bug. If that's not the case, please explain how this is a charm issue and re-open.

Changed in charm-rabbitmq-server:
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.