Rabbitmq Statefulset podManagementPolicy

Bug #1828008 reported by yann degat
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
openstack-helm-infra
New
Undecided
yann degat

Bug Description

Recently, the podManagementPolicy in statefulSets have been set to "Parallel" mode
instead of the default "OrderedReady" mode, in rabbitmq & mariadb helm packages.

(see commit: https://github.com/openstack/openstack-helm-infra/commit/b5e8c41d2ec362bfe22600e4f5623cf514c5b56f)

It can leads to race conditions issues, especially in the rabbitmq case where, if ever nodes are launched in parallel but havent reached a certain state, they can discover each other but fail to form a cluster.

(doc: https://www.rabbitmq.com/cluster-formation.html#peer-discovery-k8s)

As this "cluster forming" step isn't retried, the pods then run in standalone mode.

It happened to me a lot when i tried to launched 2 nodes clusters for tests purposes.
Dont know why it doesn't happened more with 3 nodes clusters.

Switching back explicitly to the default "OrderedReady" value of podManagementPolicy solved the issue as the second nodes starts once the first one is up & ready.

Changed in openstack-helm-infra:
assignee: nobody → yann degat (yann-degat-z)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.