SRIOV: agent may not register VFs

Bug #1831622 reported by Oleg Bondarev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Oleg Bondarev

Bug Description

When a VM instantiated with a PF-PT (direct-physical) port, the Neutron SR-IOV agent removes the respective embedded switch device instance from the switch manager. After the VM releases the PF, the associated device (sys/class/net/<device>) appears immediately, but the initialization of its VFs and the creation of the appropriate sysfs entries (/sys/class/net/<device>/device/virtfn<#vf>) may even take more than a second, depending on the platform and the NIC's kernel driver capabilities. The Neutron SR-IOV agent eagerly tries to discover and register NIC devices, that are not blacklisted and not yet known, by creating the respective embedded switch instances and enumerating the avalable VFs underneat them. However, when it is done in an early phase, where the sysfs entries for the VFs are not yet present, because the PF has just been released, then a port-less embedded switch will be created to represent that device. As a consequence, port updates that target VFs which are supposed to belong to a incorrectly registered embedded device, won't be treated properly by the agent, causing a VM instantiation timeout.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/663031

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → Medium
tags: added: pike-backport-potential queens-backport-potential rocky-backport-potential stein-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/663031
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c148c6df468b3ee1a274f191fb4317aef2e4b225
Submitter: Zuul
Branch: master

commit c148c6df468b3ee1a274f191fb4317aef2e4b225
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 4 12:57:48 2019 +0400

    SRIOV agent: wait VFs initialization on embedded switch create

    Check for configured and actual number of VFs to prevent
    device registaration with 0 VFs.

    Closes-Bug: #1831622
    Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/664205

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/664206

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/664207

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/664208

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/queens)

Reviewed: https://review.opendev.org/664207
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0fc1d32e529628d7079db4302b052d52044fcf9a
Submitter: Zuul
Branch: stable/queens

commit 0fc1d32e529628d7079db4302b052d52044fcf9a
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 4 12:57:48 2019 +0400

    SRIOV agent: wait VFs initialization on embedded switch create

    Check for configured and actual number of VFs to prevent
    device registaration with 0 VFs.

    Closes-Bug: #1831622
    Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
    (cherry picked from commit c148c6df468b3ee1a274f191fb4317aef2e4b225)

tags: added: in-stable-queens
tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/rocky)

Reviewed: https://review.opendev.org/664206
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b681fe6754e31ac1afaf68ccf61648120f42ffe9
Submitter: Zuul
Branch: stable/rocky

commit b681fe6754e31ac1afaf68ccf61648120f42ffe9
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 4 12:57:48 2019 +0400

    SRIOV agent: wait VFs initialization on embedded switch create

    Check for configured and actual number of VFs to prevent
    device registaration with 0 VFs.

    Closes-Bug: #1831622
    Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
    (cherry picked from commit c148c6df468b3ee1a274f191fb4317aef2e4b225)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/664205
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a48f157db5f6c49c1f1a6bd50c3396a7b7164085
Submitter: Zuul
Branch: stable/stein

commit a48f157db5f6c49c1f1a6bd50c3396a7b7164085
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 4 12:57:48 2019 +0400

    SRIOV agent: wait VFs initialization on embedded switch create

    Check for configured and actual number of VFs to prevent
    device registaration with 0 VFs.

    Closes-Bug: #1831622
    Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
    (cherry picked from commit c148c6df468b3ee1a274f191fb4317aef2e4b225)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/pike)

Reviewed: https://review.opendev.org/664208
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9795510c71babfee082d72453bfca2b5d4f1700d
Submitter: Zuul
Branch: stable/pike

commit 9795510c71babfee082d72453bfca2b5d4f1700d
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 4 12:57:48 2019 +0400

    SRIOV agent: wait VFs initialization on embedded switch create

    Check for configured and actual number of VFs to prevent
    device registaration with 0 VFs.

    Closes-Bug: #1831622
    Change-Id: Ie699d245f8ae2fc1d16b96432d2962788d9dba57
    (cherry picked from commit c148c6df468b3ee1a274f191fb4317aef2e4b225)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 13.0.4

This issue was fixed in the openstack/neutron 13.0.4 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 14.0.2

This issue was fixed in the openstack/neutron 14.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.1.0

This issue was fixed in the openstack/neutron 12.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 15.0.0.0b1

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron pike-eol

This issue was fixed in the openstack/neutron pike-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.