os-brick

find_multipath_device_path being called needlessly by linuxscsi.extend_volume

Bug #1832247 reported by Lee Yarwood on 2019-06-10

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	os-brick	Fix Released	Undecided	Lee Yarwood

Bug Description

This results in a delay between the SCSI rescan and domain disk block resize being called:

https://github.com/openstack/os-brick/blob/94312bd90fe42a870b4f098e00ddd9ff2aacd1bb/os_brick/initiator/linuxscsi.py#L564-L581

This is currently allowing a race between the kernel, QEMU, libvirtd and n-cpu to lead to failures when running the tempest.api.volume.test_volumes_extend.VolumesExtendAttachedTest.test_extend_attached_volume test using the Q35 machine type as seen below:

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/screen-n-cpu.txt.gz?#_Jun_09_14_51_08_780983

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/libvirt/libvirtd_log.txt.gz#_2019-06-09_14_51_08_215

http://logs.openstack.org/87/662887/19/check/tempest-full-py3/e0caad1/controller/logs/screen-n-cpu.txt.gz?#_Jun_09_14_51_20_840546

While removing this delay might workaround the above race it isn't the underlying cause.

Tags:

Lee Yarwood (lyarwood) on 2019-06-11

Changed in os-brick:
assignee:	nobody → Lee Yarwood (lyarwood)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-06-12: Fix merged to os-brick (master)

Reviewed: https://review.opendev.org/664418
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=9cf135c68640f81838e2898b4aef3e0445770089
Submitter: Zuul
Branch: master

commit 9cf135c68640f81838e2898b4aef3e0445770089
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    This change corrects this by passing down the existing use_multipath
    attribute from the iSCSI and FC connectors into linuxscsi.extend_volume.
    The same attribute is introduced to the NVMe connector to also allow it
    to skip this search for multipath devices.

Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
Closes-bug: #1832247

Changed in os-brick:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-07-11: Fix included in openstack/os-brick 2.9.1

This issue was fixed in the openstack/os-brick 2.9.1 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-11: Fix proposed to os-brick (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/698546

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-11: Fix proposed to os-brick (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/698547

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-11: Fix proposed to os-brick (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/698549

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-20: Fix merged to os-brick (stable/stein)

Reviewed: https://review.opendev.org/698546
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=f7e06bce68519ab1715a70da91f9cf4f457d3c19
Submitter: Zuul
Branch: stable/stein

commit f7e06bce68519ab1715a70da91f9cf4f457d3c19
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)

tags:

added: in-stable-stein

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-21: Fix merged to os-brick (stable/rocky)

Reviewed: https://review.opendev.org/698547
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=3eee28ce33df69cb4605338f140d2fa6e22bcd88
Submitter: Zuul
Branch: stable/rocky

commit 3eee28ce33df69cb4605338f140d2fa6e22bcd88
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)

tags:

added: in-stable-rocky

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-02-18: Fix merged to os-brick (stable/queens)

Reviewed: https://review.opendev.org/698549
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=4f0d6cebc1557f82c2138d74390888265ef28ec1
Submitter: Zuul
Branch: stable/queens

commit 4f0d6cebc1557f82c2138d74390888265ef28ec1
Author: Lee Yarwood <email address hidden>
Date: Mon Jun 10 19:02:19 2019 +0100

linuxscsi: Stop waiting for multipath devices during extend_volume

    Previously linuxscsi.extend_volume would always attempt to wait for
    multipath devices to appear regardless of multipath actually being used
    by the connector.

Conflicts:
os_brick/initiator/connectors/nvme.py

NOTE(lyarwood): Due to I6b70140be7023770b83603de723d6d2de3ebb747 not
being present in stable/queens.

    Change-Id: I29d65ae036957f3a63cba93dd330b14e3361a1b9
    Closes-bug: #1832247
    (cherry picked from commit 9cf135c68640f81838e2898b4aef3e0445770089)
    (cherry picked from commit 3eee28ce33df69cb4605338f140d2fa6e22bcd88)