Using macvtap vnic_type is not working with vif_type=hw_veb

Bug #1370348 reported by Itzik Brown
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Baodong (Robert) Li
Juno
Fix Released
Medium
Artom Lifshitz

Bug Description

When trying to boot an instance with a port using vnic_type=macvtap and vif_type=hw_veb I get this error in Compute log:

TRACE nova.compute.manager mlibvirtError: unsupported configuration: an interface of type 'direct' is requesting a vlan tag, but that is not supported for this type of connection

Revision history for this message
Daniel Berrange (berrange) wrote :

Can you say which version of libvirt you are using here.

Revision history for this message
Daniel Berrange (berrange) wrote :

So this Nova code appears to be just be plain broken & can't ever have worked in the config you request

Libvirt code has this validation check

    if (virDomainNetGetActualVlan(iface)) {
        /* vlan configuration via libvirt is only supported for
         * PCI Passthrough SR-IOV devices and openvswitch bridges.
         * otherwise log an error and fail
         */
        if (!(actualType == VIR_DOMAIN_NET_TYPE_HOSTDEV ||
              (actualType == VIR_DOMAIN_NET_TYPE_BRIDGE &&
               virtport && virtport->virtPortType
               == VIR_NETDEV_VPORT_PROFILE_OPENVSWITCH))) {
            if (netdef) {
                virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
                               _("an interface connecting to network '%s' "
                                 "is requesting a vlan tag, but that is not "
                                 "supported for this type of network"),
                               netdef->name);
            } else {
                virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
                               _("an interface of type '%s' "
                                 "is requesting a vlan tag, but that is not "
                                 "supported for this type of connection"),
                               virDomainNetTypeToString(iface->type));
            }
            goto error;

The 'hw_veb' VIF type has two ways to be configured, either using 'hostdev' mode or 'direct' mode. Only the 'hostdev' mode will support use of a vlan, but Nova configures it unconditionally even for 'direct' mode.

One possible fix is to change

def set_vif_host_backend_hw_veb(conf, net_type, devname, vlan,
                                tapname=None):
    """Populate a LibvirtConfigGuestInterface instance
    with host backend details for an device that supports hardware
    virtual ethernet bridge.
    """

    conf.net_type = net_type
    if net_type == 'direct':
        conf.source_mode = 'passthrough'
        conf.source_dev = pci_utils.get_ifname_by_pci_address(devname)
        conf.driver_name = 'vhost'
    else:
        conf.source_dev = devname
        conf.model = None
    conf.vlan = vlan
    if tapname:
        conf.target_dev = tapname

To be

def set_vif_host_backend_hw_veb(conf, net_type, devname, vlan,
                                tapname=None):
    """Populate a LibvirtConfigGuestInterface instance
    with host backend details for an device that supports hardware
    virtual ethernet bridge.
    """

    conf.net_type = net_type
    if net_type == 'direct':
        conf.source_mode = 'passthrough'
        conf.source_dev = pci_utils.get_ifname_by_pci_address(devname)
        conf.driver_name = 'vhost'
       conf.vlan = vlan
    else:
        conf.source_dev = devname
        conf.model = None

    if tapname:
        conf.target_dev = tapname

ie, only set the vlan attribute for the hostdev mode.

I've no idea if such a fix is *semantically* correct though in terms of what Neutron expects.

Could you try such a change though and report back if it works correctly.

Sean Dague (sdague)
Changed in nova:
status: New → Incomplete
Revision history for this message
Itzik Brown (itzikb1) wrote :

libvirt version is 1.2.2

Changed to:
def set_vif_host_backend_hw_veb(conf, net_type, devname, vlan,
                                tapname=None):
    """Populate a LibvirtConfigGuestInterface instance
    with host backend details for an device that supports hardware
    virtual ethernet bridge.
    """

    conf.net_type = net_type
    if net_type == 'direct':
        conf.source_mode = 'passthrough'
        conf.source_dev = pci_utils.get_ifname_by_pci_address(devname)
        conf.driver_name = 'vhost'
    else:
        conf.source_dev = devname
        conf.model = None
        conf.vlan = vlan
    if tapname:
        conf.target_dev = tapname

And it works.
Can I push this fix?

There is a need to add a the VLAN set in the Neutron side (i.e. Neutron Agent).

Revision history for this message
Baodong (Robert) Li (baoli) wrote :

Yes, I think moving 'conf.vlan = vlan' inside the else: is correct.

In the case of macvtap, the vlan interface needs to be created. This doesn't have to be done by neutron l2 agent, and can be achieved inside the vif driver. Check nova/network/linux_net.py, in which you can find how vlan interfaces are created for linux bridge (ensure_vlan()). However, it's not written as a generic routine such as create_tap_dev() that is inside the same module.

Can we create a method in linux_net.py: create_vlan_interface(vlan_num, parent_interface, mac_addr, mtu), which then can be called by ensure_vlan() as well. This routine can be called by the vif driver when setting up macvtap for hw_veb.

Itzik Brown (itzikb1)
Changed in nova:
status: Incomplete → Confirmed
Revision history for this message
Irena Berezovsky (irenab) wrote :

I am not sure if solution proposed by Robert is aligned with nova/ neutron separation of concerns. AFAIK neutron is expected to apply networking configuration to vnics created by nova. But following this, neutron agent will be required, while today there is an option to use SR-IOV without running neutron agent, i.e. Cisco VM-FEX. Please advise.

Revision history for this message
Baodong (Robert) Li (baoli) wrote :

Irena, if you check the method plug_ovs_hybrid() in vif.py, you will see that the vif driver creates a linux bridge, and adds ports in the integration bridge. Neutron later will add flows in the integration bridge and the downstream bridge to complete the data path setup (as done by the neutron agent).

Certainly, there is no integration bridge anymore for SR-IOV. But it doesn't mean that nova shouldn't participate in the data path setup any more, IMO. macvtap requires a host networking interface (be it regular ethernet interface, or vlan interface) to be connected with, and such an interface either already exists or can be created/setup by nova.

no longer affects: neutron
Revision history for this message
Daniel Berrange (berrange) wrote :

I don't see a particular problem with nova creating the vlan device if that's what's needed

Changed in nova:
assignee: nobody → Baodong (Robert) Li (baoli)
tags: added: juno-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/126357

Changed in nova:
status: Confirmed → In Progress
tags: added: libvirt
Changed in nova:
importance: Undecided → Medium
Thierry Carrez (ttx)
tags: added: juno-backport-potential
removed: juno-rc-potential
tags: added: pci-passthrough
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/126357
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=386e38198d63cf2dc45507ca7e4dc82f0bcd9bb9
Submitter: Jenkins
Branch: master

commit 386e38198d63cf2dc45507ca7e4dc82f0bcd9bb9
Author: Robert Li <email address hidden>
Date: Mon Oct 6 12:48:07 2014 -0400

    Support macvtap for vif_type being hw_veb

    This patch programs VLAN into a VF pci device, and properly
    plug/unplug it into the instance.

    Change-Id: I85eb3c1347d057d1f292e747f950065b8f394147
    Closes-Bug: 1370348
    Co-Authored-By: Itzik Brown <email address hidden>

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → kilo-1
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/juno)

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/151391

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/juno)

Reviewed: https://review.openstack.org/151391
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4ea3c18b2a430d136a9826a086376526a85f4430
Submitter: Jenkins
Branch: stable/juno

commit 4ea3c18b2a430d136a9826a086376526a85f4430
Author: Robert Li <email address hidden>
Date: Mon Oct 6 12:48:07 2014 -0400

    Support macvtap for vif_type being hw_veb

    This patch programs VLAN into a VF pci device, and properly
    plug/unplug it into the instance.

    Change-Id: I85eb3c1347d057d1f292e747f950065b8f394147
    Closes-Bug: 1370348
    Co-Authored-By: Itzik Brown <email address hidden>
    (cherry picked from commit 386e38198d63cf2dc45507ca7e4dc82f0bcd9bb9)

tags: added: in-stable-juno
Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.