haproxy needs to be configured to avoid deadlock with mysql galera cluster

Bug #1387685 reported by Tom Howley on 2014-10-30
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
tripleo
High
Nicholas Randon

Bug Description

When the overcloud controller is deployed in HA mode and mysql galera cluster is being used, we have a potential deadlock issue when two mysql instances are writing at the same time. One issue is that mysql galera does not support SELECT...FOR UPDATE calls as it has no concept of cross-node locking.

I'm not aware if this issue has changed. If not, the workaround is configure haproxy to effectively use mysql in active-passive mode, which can be achieved by configuring all but one mysql instances as 'backup' nodes.

Changed in tripleo:
assignee: nobody → Nicholas Randon (nicholas-randon)

Fix proposed to branch: master
Review: https://review.openstack.org/132048

Changed in tripleo:
status: New → In Progress
Changed in tripleo:
assignee: Nicholas Randon (nicholas-randon) → Alexis Lee (alexisl)

Fix proposed to branch: master
Review: https://review.openstack.org/132059

Changed in tripleo:
assignee: Alexis Lee (alexisl) → Nicholas Randon (nicholas-randon)
Clint Byrum (clint-fewbar) wrote :

Marking this as "High" because it involves spurious database errors. Ideally OpenStack would handle rollbacks by retrying, but we know that just isn't the case.

Changed in tripleo:
importance: Undecided → High
Changed in tripleo:
status: In Progress → Invalid

Change abandoned by Nicholas Randon (<email address hidden>) on branch: master
Review: https://review.openstack.org/132048

Change abandoned by Nicholas Randon (<email address hidden>) on branch: master
Review: https://review.openstack.org/132059

Tom Howley (tom-howley) wrote :

This bug has been marked as invalid as the existing config marks all nodes in the haproxy pool as backup, thereby disabling load-balancing, as haproxy will just always just pick the first available node in the list and only change when that node fails. As a result, the existing config already minimises chance of deadlock in galera cluster.

(Please correct if I'm wrong on this.)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers