find_multipath_device_path being called needlessly by linuxscsi.extend_volume

Bug #1832247 reported by Lee Yarwood
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-brick
Fix Released
Undecided
Lee Yarwood

Bug Description

This results in a delay between the SCSI rescan and domain disk block resize being called:

https://github.com/openstack/os-brick/blob/94312bd90fe42a870b4f098e00ddd9ff2aacd1bb/os_brick/initiator/linuxscsi.py#L564-L581

This is currently allowing a race between the kernel, QEMU, libvirtd and n-cpu to lead to failures when running the tempest.api.volume.test_volumes_extend.VolumesExtendAttachedTest.test_extend_attached_volume test using the Q35 machine type as seen below:

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/screen-n-cpu.txt.gz?#_Jun_09_14_51_08_780983

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/libvirt/libvirtd_log.txt.gz#_2019-06-09_14_51_08_215

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/screen-n-cpu.txt.gz?#_Jun_09_14_51_20_840546

While removing this delay might workaround the above race it isn't the underlying cause.

Lee Yarwood (lyarwood)
Changed in os-brick:
assignee: nobody → Lee Yarwood (lyarwood)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (master)

Reviewed: https://review.opendev.org/664418
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=9cf135c68640f81838e2898b4aef3e0445770089
Submitter: Zuul
Branch: master

commit 9cf135c68640f81838e2898b4aef3e0445770089
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

    linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    This change corrects this by passing down the existing use_multipath
    attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
    The same attribute is introduced to the NVMe connector to also allow it
    to skip this search for multipath devices.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247

Changed in os-brick:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.9.1

This issue was fixed in the openstack/os-brick 2.9.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/698546

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/698547

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-brick (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/698549

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/stein)

Reviewed: https://review.opendev.org/698546
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=f7e06bce68519ab1715a70da91f9cf4f457d3c19
Submitter: Zuul
Branch: stable/stein

commit f7e06bce68519ab1715a70da91f9cf4f457d3c19
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

    linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    This change corrects this by passing down the existing use_multipath
    attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
    The same attribute is introduced to the NVMe connector to also allow it
    to skip this search for multipath devices.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/rocky)

Reviewed: https://review.opendev.org/698547
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=3eee28ce33df69cb4605338f140d2fa6e22bcd88
Submitter: Zuul
Branch: stable/rocky

commit 3eee28ce33df69cb4605338f140d2fa6e22bcd88
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

    linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    This change corrects this by passing down the existing use_multipath
    attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
    The same attribute is introduced to the NVMe connector to also allow it
    to skip this search for multipath devices.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-brick (stable/queens)

Reviewed: https://review.opendev.org/698549
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=4f0d6cebc1557f82c2138d74390888265ef28ec1
Submitter: Zuul
Branch: stable/queens

commit 4f0d6cebc1557f82c2138d74390888265ef28ec1
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

    linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    This change corrects this by passing down the existing use_multipath
    attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
    The same attribute is introduced to the NVMe connector to also allow it
    to skip this search for multipath devices.

    Conflicts:
        os_brick/initiator/connectors/nvme.py

    NOTE(lyarwood): Due to I6b70140be7023770b83603de723d6d2de3ebb747 not
    being present in stable/queens.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)
    (cherry picked from commit 3eee28ce33df69cb4605338f140d2fa6e22bcd88)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.5.10

This issue was fixed in the openstack/os-brick 2.5.10 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick 2.8.4

This issue was fixed in the openstack/os-brick 2.8.4 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-brick queens-eol

This issue was fixed in the openstack/os-brick queens-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.