There's a potential deadlock scenario in PowerMax's masking.py
"do_move_volume_between_storage_groups" method.
The method uses 2 locks, one for the source Storage Group and another
for the destination Storage Group, and it could happen that if 2
requests going in opposite directions are received simultaneously their
first lock acquisition interleaves resulting in a deadlock situation.
- User requests an instance migration from A to B
- User requests an instance migration from B to A
- Driver acquires the first lock for A-to-B for example something like
cinder-emc-sg-SGA-###
- Driver acquires the first lock for B-to-A for example something like
cinder-emc-sgSGB-###
The deadlock happens because A-to-B waits forever for the lock held by
the B-to-A operation, which in turn cannot proceed because it’s waiting
for lock held by A-to-B.
This patch fixes it using the new coordination.synchronized
functionality that ensures that a series of locks are always acquired in
the same order, preventing deadlocks.
Reviewed: https:/ /review. opendev. org/c/openstack /cinder/ +/848900 /opendev. org/openstack/ cinder/ commit/ 411852892d803b6 06110d0956a5976 4925c16ec6
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 411852892d803b6 06110d0956a5976 4925c16ec6
Author: Gorka Eguileor <email address hidden>
Date: Wed Jul 6 20:51:34 2022 +0200
PowerMax: Fix deadlock moving SGs
There's a potential deadlock scenario in PowerMax's masking.py move_volume_ between_ storage_ groups" method.
"do_
The method uses 2 locks, one for the source Storage Group and another
for the destination Storage Group, and it could happen that if 2
requests going in opposite directions are received simultaneously their
first lock acquisition interleaves resulting in a deadlock situation.
def do_move_
The scenario would be like this:
- User requests an instance migration from A to B emc-sg- SGA-### emc-sgSGB- ###
- User requests an instance migration from B to A
- Driver acquires the first lock for A-to-B for example something like
cinder-
- Driver acquires the first lock for B-to-A for example something like
cinder-
The deadlock happens because A-to-B waits forever for the lock held by
the B-to-A operation, which in turn cannot proceed because it’s waiting
for lock held by A-to-B.
This patch fixes it using the new coordination. synchronized
functionality that ensures that a series of locks are always acquired in
the same order, preventing deadlocks.
Closes-Bug: #1980870 edcf45d73ab3a21 5976d3fac3a
Change-Id: I7eda4645575cfa