iSCSI Multipath may leave residual paths due to race condition

Bug #1664032 reported by Gorka Eguileor
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
os-brick
Fix Released
Undecided
Gorka Eguileor

Bug Description

iSCSI multipath rescan uses iscsiadm --rescan option for nodes and sessions, which can end up recreating devices that had just been removed if there's a race condition between the removal of a SCSI device and the connection of a volume.

The race condition happens if a rescan done when attaching happens right between us removing the path and removing the exported lun, because the rescan will add not only the new path we are attaching, but the old path we are removing, since the lun still hasn't been removed.

This would leave orphaned devices that unnecessarily pollute our environment.

Gorka Eguileor (gorka)
Changed in os-brick:
assignee: nobody → Gorka Eguileor (gorka)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (master)

Fix proposed to branch: master
Review: https://review.openstack.org/433104

Changed in os-brick:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (master)

Reviewed: https://review.openstack.org/433104
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=a6e789f27edc5cae84a786f3d3604cc6546820d7
Submitter: Jenkins
Branch: master

commit a6e789f27edc5cae84a786f3d3604cc6546820d7
Author: Gorka Eguileor <email address hidden>
Date: Sun Feb 12 01:58:46 2017 +0100

    Fix iSCSI multipath rescan

    iSCSI multipath rescan uses iscsiadm --rescan option for nodes and
    sessions, which can end up recreating devices that had just been removed
    if there's a race condition between the removal of a SCSI device and the
    connection of a volume.

    The race condition happens if a rescan done when attaching happens right
    between us removing the path and removing the exported lun, because the
    rescan will add not only the new path we are attaching, but the old path
    we are removing, since the lun still hasn't been removed.

    This would leave orphaned devices that unnecessarily pollute our
    environment,

    This patch narrows the rescan to only rescan for the specific target id,
    channel, and lun number if we can find this information.

    When we cannot find this information we do the scan as we were doing it
    before.

    Closes-Bug: #1664032
    Change-Id: I1b3bd34db260165a6ea9ca061f946d6dfcf8553f

Changed in os-brick:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/445943

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 1.12.0

This issue was fixed in the openstack/os-brick 1.12.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on os-brick (stable/ocata)

Change abandoned by Sean McGinnis (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/445943
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.