failed to boot guest with vnic_type direct when rx_queue_size, tx_queue_size and hw_vif_type are set

Bug #1797146 reported by Stephen Finucane
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Stephen Finucane

Bug Description

Bug #1789074 addressed an issue with booting a guest with vnic_type direct when rx_queue_size and tx_queue_size. However, this failed to address an additional permutation: the user specifying hw_vif_type=virtio. If the user does this, the problem occurs once again.

Reproduction steps are the same noted in bug #1789074 with one additional step needed:

  openstack image set --property hw_vif_type=virtio $IMAGE

Once configured, boot an instance with this image and an SRIOV (PF or VF) interface and the instance will fail to spawn. This is because we first read and set the VIF model from the image metadata property:

  https://github.com/openstack/nova/blob/622ebf2fab0a9bf75ee12437bef28f60e083f849/nova/virt/libvirt/vif.py#L134-L135

Which means a later check passes:

  https://github.com/openstack/nova/blob/622ebf2fab0a9bf75ee12437bef28f60e083f849/nova/virt/libvirt/vif.py#L172

Without setting this property, that check would fail as we never configure the model for direct SR-IOV interfaces.

  https://github.com/openstack/nova/blob/622ebf2fab0a9bf75ee12437bef28f60e083f849/nova/virt/libvirt/vif.py#L139

Revision history for this message
Stephen Finucane (stephenfinucane) wrote :

The obvious fix would be to only retrieve the vif type from the image if the VIF is not a direct passthrough. However, there might be more complications to it than this (some SR-IOV NICs provide a virtio interface, I'm told).

Changed in nova:
status: New → Confirmed
importance: Undecided → Medium
milestone: none → stein-1
assignee: nobody → Stephen Finucane (stephenfinucane)
tags: added: libvirt neutron pci
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/609460

Changed in nova:
status: Confirmed → In Progress
Matt Riedemann (mriedem)
Changed in nova:
milestone: stein-1 → none
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/609460
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a235451578ad7af25268e82500dff5ba6a64ea4e
Submitter: Zuul
Branch: master

commit a235451578ad7af25268e82500dff5ba6a64ea4e
Author: Stephen Finucane <email address hidden>
Date: Wed Oct 10 16:29:40 2018 +0100

    Ignore hw_vif_type for direct, direct-physical vNIC types

    As part of the fix for bug #1789074, we stopped setting the VIF model to
    virtio for a vif_type of 'direct' or 'direct-physical'. However, if
    users configured the 'hw_vif_type' image metadata property, the VIF
    model would be unconditionally set to whatever the user provided. This
    would result in bug #1789074 recurring. Given that the model attribute
    is unnecessary for these vNIC types, we can resolve this issue by
    simply ignoring the image metadata property when configuring such VIFs.

    Change-Id: I3bf5587c3a49c807bb14533dbbea8f72dca2f02b
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-bug: #1797146

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/667354

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/667355

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/stein)

Reviewed: https://review.opendev.org/667354
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d78310e67a8fcf59465907eddecba4f5145a0a6f
Submitter: Zuul
Branch: stable/stein

commit d78310e67a8fcf59465907eddecba4f5145a0a6f
Author: Stephen Finucane <email address hidden>
Date: Wed Oct 10 16:29:40 2018 +0100

    Ignore hw_vif_type for direct, direct-physical vNIC types

    As part of the fix for bug #1789074, we stopped setting the VIF model to
    virtio for a vif_type of 'direct' or 'direct-physical'. However, if
    users configured the 'hw_vif_type' image metadata property, the VIF
    model would be unconditionally set to whatever the user provided. This
    would result in bug #1789074 recurring. Given that the model attribute
    is unnecessary for these vNIC types, we can resolve this issue by
    simply ignoring the image metadata property when configuring such VIFs.

    Change-Id: I3bf5587c3a49c807bb14533dbbea8f72dca2f02b
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-bug: #1797146
    (cherry picked from commit a235451578ad7af25268e82500dff5ba6a64ea4e)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.opendev.org/667355
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0142cc840f6680e5011381c6793173ebd0c9ee1e
Submitter: Zuul
Branch: stable/rocky

commit 0142cc840f6680e5011381c6793173ebd0c9ee1e
Author: Stephen Finucane <email address hidden>
Date: Wed Oct 10 16:29:40 2018 +0100

    Ignore hw_vif_type for direct, direct-physical vNIC types

    As part of the fix for bug #1789074, we stopped setting the VIF model to
    virtio for a vif_type of 'direct' or 'direct-physical'. However, if
    users configured the 'hw_vif_type' image metadata property, the VIF
    model would be unconditionally set to whatever the user provided. This
    would result in bug #1789074 recurring. Given that the model attribute
    is unnecessary for these vNIC types, we can resolve this issue by
    simply ignoring the image metadata property when configuring such VIFs.

    Change-Id: I3bf5587c3a49c807bb14533dbbea8f72dca2f02b
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-bug: #1797146
    (cherry picked from commit a235451578ad7af25268e82500dff5ba6a64ea4e)
    (cherry picked from commit d78310e67a8fcf59465907eddecba4f5145a0a6f)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.0.2

This issue was fixed in the openstack/nova 19.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.2.2

This issue was fixed in the openstack/nova 18.2.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 20.0.0.0rc1

This issue was fixed in the openstack/nova 20.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.