os-brick

FC scan always scanning all HBAs when receiving initiator_target_map

Bug #1828440 reported by Gorka Eguileor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-brick
Fix Released
Undecided
Gorka Eguileor

Bug Description

Current FC OS-Brick code only checks for single WWNN to exclude some HBAs from scanning when we don't receive an initiator_target_map from the backend.

There are storage arrays,like XtremIO, that due to their architecture and design have all target ports automatically mapped to LUNs and are returning the initiator_target_map, but some of those ports may not be connected to our HBAs, so we end up with another case of bug #1765000.

We should always check if the target implements single WWNN, not only when we don't receive the initiator_target_map.

Doing that we decrease the likelihood of ending with unexpected devices in our system, because now we will ignore unconnected HBAs (even if reported in the initiator_target_map) if we are working with single WWNN target.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (master)

Fix proposed to branch: master
Review: https://review.opendev.org/658154

Changed in os-brick:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (master)

Reviewed: https://review.opendev.org/658154
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=70899a9aa3e4bdd3f1b1a7024807390cd55c4958
Submitter: Zuul
Branch: master

commit 70899a9aa3e4bdd3f1b1a7024807390cd55c4958
Author: Gorka Eguileor <email address hidden>
Date: Thu May 9 17:25:03 2019 +0200

    FC: Ignore some HBAs from map for single WWNN

    Current FC OS-Brick code only checks for single WWNN to exclude some
    HBAs from scanning when we don't receive an initiator_target_map from
    the backend.

    There are storage arrays,like XtremIO, that due to their architecture
    and design have all target ports automatically mapped to LUNs and are
    returning the initiator_target_map, but some of those ports may not be
    connected to our HBAs, so we end up with another case of bug #1765000.

    This patch makes sure that we always check if the target implements
    single WWNN, not only when we don't receive the initiator_target_map.

    With this we decrease the likelihood of ending with unexpected devices
    in our system, because now we will ignore unconnected HBAs (even if
    reported in the initiator_target_map) if we are working with single WWNN
    target.

    Related-Bug: #1765000
    Closes-Bug: #1828440
    Change-Id: I02c208142f5b342f894666831449243863eccbfe

Changed in os-brick:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/665641

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/stein)

Reviewed: https://review.opendev.org/665641
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=1ee6acb509910d7de1213fb71cd0d6166991609a
Submitter: Zuul
Branch: stable/stein

commit 1ee6acb509910d7de1213fb71cd0d6166991609a
Author: Gorka Eguileor <email address hidden>
Date: Thu May 9 17:25:03 2019 +0200

    FC: Ignore some HBAs from map for single WWNN

    Current FC OS-Brick code only checks for single WWNN to exclude some
    HBAs from scanning when we don't receive an initiator_target_map from
    the backend.

    There are storage arrays,like XtremIO, that due to their architecture
    and design have all target ports automatically mapped to LUNs and are
    returning the initiator_target_map, but some of those ports may not be
    connected to our HBAs, so we end up with another case of bug #1765000.

    This patch makes sure that we always check if the target implements
    single WWNN, not only when we don't receive the initiator_target_map.

    With this we decrease the likelihood of ending with unexpected devices
    in our system, because now we will ignore unconnected HBAs (even if
    reported in the initiator_target_map) if we are working with single WWNN
    target.

    Related-Bug: #1765000
    Closes-Bug: #1828440
    Change-Id: I02c208142f5b342f894666831449243863eccbfe
    (cherry picked from commit 70899a9aa3e4bdd3f1b1a7024807390cd55c4958)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/667507

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/668567

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/rocky)

Reviewed: https://review.opendev.org/667507
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=64cbaaea47822dff3ae5fbd104f9bff8fe79d08d
Submitter: Zuul
Branch: stable/rocky

commit 64cbaaea47822dff3ae5fbd104f9bff8fe79d08d
Author: Gorka Eguileor <email address hidden>
Date: Thu May 9 17:25:03 2019 +0200

    FC: Ignore some HBAs from map for single WWNN

    Current FC OS-Brick code only checks for single WWNN to exclude some
    HBAs from scanning when we don't receive an initiator_target_map from
    the backend.

    There are storage arrays,like XtremIO, that due to their architecture
    and design have all target ports automatically mapped to LUNs and are
    returning the initiator_target_map, but some of those ports may not be
    connected to our HBAs, so we end up with another case of bug #1765000.

    This patch makes sure that we always check if the target implements
    single WWNN, not only when we don't receive the initiator_target_map.

    With this we decrease the likelihood of ending with unexpected devices
    in our system, because now we will ignore unconnected HBAs (even if
    reported in the initiator_target_map) if we are working with single WWNN
    target.

    Related-Bug: #1765000
    Closes-Bug: #1828440
    Change-Id: I02c208142f5b342f894666831449243863eccbfe
    (cherry picked from commit 70899a9aa3e4bdd3f1b1a7024807390cd55c4958)
    (cherry picked from commit 1ee6acb509910d7de1213fb71cd0d6166991609a)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.9.1

This issue was fixed in the openstack/os-brick 2.9.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.8.2

This issue was fixed in the openstack/os-brick 2.8.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.5.8

This issue was fixed in the openstack/os-brick 2.5.8 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/queens)

Reviewed: https://review.opendev.org/668567
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=66d9d2a908e9baeb36662a149c24c887bbd6ba2b
Submitter: Zuul
Branch: stable/queens

commit 66d9d2a908e9baeb36662a149c24c887bbd6ba2b
Author: Gorka Eguileor <email address hidden>
Date: Thu May 9 17:25:03 2019 +0200

    FC: Ignore some HBAs from map for single WWNN

    Current FC OS-Brick code only checks for single WWNN to exclude some
    HBAs from scanning when we don't receive an initiator_target_map from
    the backend.

    There are storage arrays,like XtremIO, that due to their architecture
    and design have all target ports automatically mapped to LUNs and are
    returning the initiator_target_map, but some of those ports may not be
    connected to our HBAs, so we end up with another case of bug #1765000.

    This patch makes sure that we always check if the target implements
    single WWNN, not only when we don't receive the initiator_target_map.

    With this we decrease the likelihood of ending with unexpected devices
    in our system, because now we will ignore unconnected HBAs (even if
    reported in the initiator_target_map) if we are working with single WWNN
    target.

    Related-Bug: #1765000
    Closes-Bug: #1828440
    Change-Id: I02c208142f5b342f894666831449243863eccbfe
    (cherry picked from commit 70899a9aa3e4bdd3f1b1a7024807390cd55c4958)
    (cherry picked from commit 1ee6acb509910d7de1213fb71cd0d6166991609a)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/690640

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.3.9

This issue was fixed in the openstack/os-brick 2.3.9 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (master)

Reviewed: https://review.opendev.org/690640
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=708733e49598145eb8e898d486723dc3df00a52f
Submitter: Zuul
Branch: master

commit 708733e49598145eb8e898d486723dc3df00a52f
Author: Gorka Eguileor <email address hidden>
Date: Mon Oct 14 16:42:22 2019 +0200

    Fix FC scan too broad

    After some changes to the FC connector we have introduced a regression
    on the way we do the scans, and we end up scanning using wildcards even
    though we shouldn't.

    The targets in the "initiator_target_map" don't mean that they are all
    connected, so we must take that into account.

    With the current code, if we have the following connections:

    HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
              \--- SWITCH ---- port X (channel 0, target 3)

    HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
              \--- SWITCH ---- port Z (channel 0, target 3)

    We will end up with the following scans 8 scans for LUN L:

      - - L > host7
      - - L > host7
      0 2 L > host7
      0 3 L > host7

      0 2 L > host8
      0 3 L > host8
      - - L > host8
      - - L > host8

    Which correspond to the responses from _get_hba_channel_scsi_target like
    this:

                   port Y port Z port W port X
      host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
      host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]

    And we should only be doing 4 scans:

      0 2 L > host7
      0 3 L > host7
      0 2 L > host8
      0 3 L > host8

    Most storage arrays get their target ports automatically detected by the
    Linux FC initiator and sysfs gets populated with that information, but
    there are some that don't.

    We'll do a narrow scan using the channel, target, and LUN for the former
    and a wider scan for the latter.

    If all paths to a former type of array were down on the system boot the
    array could look like it's of the latter type and make us bring us
    unwanted volumes into the system by doing a broad scan.

    To prevent this from happening Cinder drivers can use the
    "enable_wildcard_scan" key in the connection information to let us know
    they don't want us to do broad scans even if no target ports are found
    (because they know the cause is there's no connection).

    Close-Bug: #1849504
    Related-Bug: #1765000
    Related-Bug: #1828440
    Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/694314

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/train)

Reviewed: https://review.opendev.org/694314
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=c76b6dc514b8e17e1211e4a7465a7e2a635adea3
Submitter: Zuul
Branch: stable/train

commit c76b6dc514b8e17e1211e4a7465a7e2a635adea3
Author: Gorka Eguileor <email address hidden>
Date: Mon Oct 14 16:42:22 2019 +0200

    Fix FC scan too broad

    After some changes to the FC connector we have introduced a regression
    on the way we do the scans, and we end up scanning using wildcards even
    though we shouldn't.

    The targets in the "initiator_target_map" don't mean that they are all
    connected, so we must take that into account.

    With the current code, if we have the following connections:

    HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
              \--- SWITCH ---- port X (channel 0, target 3)

    HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
              \--- SWITCH ---- port Z (channel 0, target 3)

    We will end up with the following scans 8 scans for LUN L:

      - - L > host7
      - - L > host7
      0 2 L > host7
      0 3 L > host7

      0 2 L > host8
      0 3 L > host8
      - - L > host8
      - - L > host8

    Which correspond to the responses from _get_hba_channel_scsi_target like
    this:

                   port Y port Z port W port X
      host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
      host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]

    And we should only be doing 4 scans:

      0 2 L > host7
      0 3 L > host7
      0 2 L > host8
      0 3 L > host8

    Most storage arrays get their target ports automatically detected by the
    Linux FC initiator and sysfs gets populated with that information, but
    there are some that don't.

    We'll do a narrow scan using the channel, target, and LUN for the former
    and a wider scan for the latter.

    If all paths to a former type of array were down on the system boot the
    array could look like it's of the latter type and make us bring us
    unwanted volumes into the system by doing a broad scan.

    To prevent this from happening Cinder drivers can use the
    "enable_wildcard_scan" key in the connection information to let us know
    they don't want us to do broad scans even if no target ports are found
    (because they know the cause is there's no connection).

    Close-Bug: #1849504
    Related-Bug: #1765000
    Related-Bug: #1828440
    Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9
    (cherry picked from commit 708733e49598145eb8e898d486723dc3df00a52f)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/stein)

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/694890

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/stein)

Reviewed: https://review.opendev.org/694890
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=8933503041529978b9fd816e1788b1b0c343b540
Submitter: Zuul
Branch: stable/stein

commit 8933503041529978b9fd816e1788b1b0c343b540
Author: Gorka Eguileor <email address hidden>
Date: Mon Oct 14 16:42:22 2019 +0200

    Fix FC scan too broad

    After some changes to the FC connector we have introduced a regression
    on the way we do the scans, and we end up scanning using wildcards even
    though we shouldn't.

    The targets in the "initiator_target_map" don't mean that they are all
    connected, so we must take that into account.

    With the current code, if we have the following connections:

    HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
              \--- SWITCH ---- port X (channel 0, target 3)

    HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
              \--- SWITCH ---- port Z (channel 0, target 3)

    We will end up with the following scans 8 scans for LUN L:

      - - L > host7
      - - L > host7
      0 2 L > host7
      0 3 L > host7

      0 2 L > host8
      0 3 L > host8
      - - L > host8
      - - L > host8

    Which correspond to the responses from _get_hba_channel_scsi_target like
    this:

                   port Y port Z port W port X
      host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
      host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]

    And we should only be doing 4 scans:

      0 2 L > host7
      0 3 L > host7
      0 2 L > host8
      0 3 L > host8

    Most storage arrays get their target ports automatically detected by the
    Linux FC initiator and sysfs gets populated with that information, but
    there are some that don't.

    We'll do a narrow scan using the channel, target, and LUN for the former
    and a wider scan for the latter.

    If all paths to a former type of array were down on the system boot the
    array could look like it's of the latter type and make us bring us
    unwanted volumes into the system by doing a broad scan.

    To prevent this from happening Cinder drivers can use the
    "enable_wildcard_scan" key in the connection information to let us know
    they don't want us to do broad scans even if no target ports are found
    (because they know the cause is there's no connection).

    Close-Bug: #1849504
    Related-Bug: #1765000
    Related-Bug: #1828440
    Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9
    (cherry picked from commit 708733e49598145eb8e898d486723dc3df00a52f)
    (cherry picked from commit c76b6dc514b8e17e1211e4a7465a7e2a635adea3)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/queens)

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/696385

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-brick (stable/rocky)

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/697133

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/rocky)

Reviewed: https://review.opendev.org/697133
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=34a3ca23b3184e9b356877432d0f44c836705a70
Submitter: Zuul
Branch: stable/rocky

commit 34a3ca23b3184e9b356877432d0f44c836705a70
Author: Gorka Eguileor <email address hidden>
Date: Mon Oct 14 16:42:22 2019 +0200

    Fix FC scan too broad

    After some changes to the FC connector we have introduced a regression
    on the way we do the scans, and we end up scanning using wildcards even
    though we shouldn't.

    The targets in the "initiator_target_map" don't mean that they are all
    connected, so we must take that into account.

    With the current code, if we have the following connections:

    HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
              \--- SWITCH ---- port X (channel 0, target 3)

    HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
              \--- SWITCH ---- port Z (channel 0, target 3)

    We will end up with the following scans 8 scans for LUN L:

      - - L > host7
      - - L > host7
      0 2 L > host7
      0 3 L > host7

      0 2 L > host8
      0 3 L > host8
      - - L > host8
      - - L > host8

    Which correspond to the responses from _get_hba_channel_scsi_target like
    this:

                   port Y port Z port W port X
      host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
      host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]

    And we should only be doing 4 scans:

      0 2 L > host7
      0 3 L > host7
      0 2 L > host8
      0 3 L > host8

    Most storage arrays get their target ports automatically detected by the
    Linux FC initiator and sysfs gets populated with that information, but
    there are some that don't.

    We'll do a narrow scan using the channel, target, and LUN for the former
    and a wider scan for the latter.

    If all paths to a former type of array were down on the system boot the
    array could look like it's of the latter type and make us bring us
    unwanted volumes into the system by doing a broad scan.

    To prevent this from happening Cinder drivers can use the
    "enable_wildcard_scan" key in the connection information to let us know
    they don't want us to do broad scans even if no target ports are found
    (because they know the cause is there's no connection).

    Close-Bug: #1849504
    Related-Bug: #1765000
    Related-Bug: #1828440
    Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9
    (cherry picked from commit 708733e49598145eb8e898d486723dc3df00a52f)
    (cherry picked from commit c76b6dc514b8e17e1211e4a7465a7e2a635adea3)
    (cherry picked from commit 8933503041529978b9fd816e1788b1b0c343b540)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-brick (stable/queens)

Reviewed: https://review.opendev.org/696385
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=2e46deeaa109f46686171e9af6ea12e94eb9e1f7
Submitter: Zuul
Branch: stable/queens

commit 2e46deeaa109f46686171e9af6ea12e94eb9e1f7
Author: Gorka Eguileor <email address hidden>
Date: Mon Oct 14 16:42:22 2019 +0200

    Fix FC scan too broad

    After some changes to the FC connector we have introduced a regression
    on the way we do the scans, and we end up scanning using wildcards even
    though we shouldn't.

    The targets in the "initiator_target_map" don't mean that they are all
    connected, so we must take that into account.

    With the current code, if we have the following connections:

    HBA host7 ---- SWITCH ---- port W (channel 0, target 2)
              \--- SWITCH ---- port X (channel 0, target 3)

    HBA host8 ---- SWITCH ---- port Y (channel 0, target 2)
              \--- SWITCH ---- port Z (channel 0, target 3)

    We will end up with the following scans 8 scans for LUN L:

      - - L > host7
      - - L > host7
      0 2 L > host7
      0 3 L > host7

      0 2 L > host8
      0 3 L > host8
      - - L > host8
      - - L > host8

    Which correspond to the responses from _get_hba_channel_scsi_target like
    this:

                   port Y port Z port W port X
      host7 ... ['-','-',L] ['-','-',L] ['0','2',L] ['0','3',L]
      host8 ... ['0','2',L] ['0','3',L] ['-','-',L] ['-','-',L]

    And we should only be doing 4 scans:

      0 2 L > host7
      0 3 L > host7
      0 2 L > host8
      0 3 L > host8

    Most storage arrays get their target ports automatically detected by the
    Linux FC initiator and sysfs gets populated with that information, but
    there are some that don't.

    We'll do a narrow scan using the channel, target, and LUN for the former
    and a wider scan for the latter.

    If all paths to a former type of array were down on the system boot the
    array could look like it's of the latter type and make us bring us
    unwanted volumes into the system by doing a broad scan.

    To prevent this from happening Cinder drivers can use the
    "enable_wildcard_scan" key in the connection information to let us know
    they don't want us to do broad scans even if no target ports are found
    (because they know the cause is there's no connection).

    Close-Bug: #1849504
    Related-Bug: #1765000
    Related-Bug: #1828440
    Change-Id: I5dbefaff43fb902b15117b443fc92f7b6a6ad8c9
    (cherry picked from commit 708733e49598145eb8e898d486723dc3df00a52f)
    (cherry picked from commit c76b6dc514b8e17e1211e4a7465a7e2a635adea3)
    (cherry picked from commit 8933503041529978b9fd816e1788b1b0c343b540)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.