Ovs-dpdk doesn't grab virtio NIC port via devstack

Bug #1704279 reported by Tetsuro Nakamura on 2017-07-14
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-ovs-dpdk
Wishlist
Tetsuro Nakamura

Bug Description

Description
===========

When you map a virtio backend NIC port to ovs-dpdk with devstack, ovs-dpdk doesn't grab the NIC port, and you get no error.

Steps to reproduce
==================

1. create devstack to map a virtio-backend port to ovs-dpdk
The local.conf includes like the following line,
```
OVS_BRIDGE_MAPPINGS=default:br-ens4
```
where ens4 is virtio backend.
```
$ sudo ethtool -i ens4
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:00:04.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
```

2. run stack.sh to see that after all ovs is't connected the NIC port
```
$ sudo ovs-vsctl show
e3f789a9-6061-4ec7-a592-06cbc7be2160
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge "br-ens4"
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "phy-br-ens4"
            Interface "phy-br-ens4"
                type: patch
                options: {peer="int-br-ens4"}
        Port "br-ens4"
            Interface "br-ens4"
                type: internal
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "int-br-ens4"
            Interface "int-br-ens4"
                type: patch
                options: {peer="phy-br-ens4"}
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
```

Expected result
===============

Devstack connects ovs to the specified port appropriately.

Environment
===========

I'm runnning devstack on ubuntu16.04 built using KVM on ubuntu16.04

Logs & Configs
==============
stack.sh.log is below
```
$ cat /opt/stack/logs/stack.sh.log | grep ens4
2017-07-13 09:43:00.089 | ++/opt/stack/networking-ovs-dpdk/devstack/settings:source:44 OVS_BRIDGE_MAPPINGS=default:br-ens4
2017-07-13 09:47:47.760 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:464 ls -al /sys/class/net/ens3 /sys/class/net/ens4 /sys/class/net/lo
2017-07-13 09:47:47.785 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:466 [[ default:br-ens4 = '' ]]
2017-07-13 09:47:47.805 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:471 OVS_BRIDGES=default:br-ens4
2017-07-13 09:47:47.839 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:474 bridge=br-ens4
2017-07-13 09:47:47.854 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:475 nic=ens4
2017-07-13 09:47:47.879 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:477 OVS_DPDK_PORT_MAPPINGS=ens4:br-ens4
2017-07-13 09:47:47.930 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:522 MAPPINGS=ens4:br-ens4
2017-07-13 09:47:47.980 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:527 nic=ens4
2017-07-13 09:47:47.991 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:528 bridge=br-ens4
2017-07-13 09:47:48.002 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:529 printf '%s in %s\n' ens4 br-ens4
2017-07-13 09:47:48.002 | ens4 in br-ens4
2017-07-13 09:47:48.046 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == ens3 ]]
2017-07-13 09:47:48.072 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 echo virtio1#ens4
2017-07-13 09:47:48.084 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == ens4 ]]
2017-07-13 09:47:48.109 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:533 NICS=virtio1#ens4
2017-07-13 09:47:48.151 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == lo ]]
2017-07-13 09:47:48.247 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:541 sudo sed 's/OVS_PCI_MAPPINGS=.*/OVS_PCI_MAPPINGS=virtio1#ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:48.275 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:542 sudo sed 's/OVS_BRIDGE_MAPPINGS=.*/OVS_BRIDGE_MAPPINGS=default:br-ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:48.302 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:543 sudo sed 's/OVS_DPDK_PORT_MAPPINGS=.*/OVS_DPDK_PORT_MAPPINGS=ens4:br-ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:49.778 | + local MAPPINGS=virtio1#ens4
2017-07-13 09:47:49.779 | ++ echo virtio1#ens4
2017-07-13 09:48:28.128 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:44 [[ default:br-ens4 == '' ]]
2017-07-13 09:48:28.137 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:50 [[ default:br-ens4 != '' ]]
2017-07-13 09:48:28.147 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:51 iniset /etc/neutron/plugins/ml2/ml2_conf.ini ovs bridge_mappings default:br-ens4
```

It is expected to be like below, where rename6 is a physical NIC port.
```
$ cat /opt/stack/logs/stack.sh.log | grep rename6
2017-07-13 01:59:58.066 | ++/opt/stack/networking-ovs-dpdk/devstack/settings:source:44 OVS_BRIDGE_MAPPINGS=default:br-rename6
2017-07-13 02:05:23.216 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:464 ls -al /sys/class/net/eno1 /sys/class/net/eno2 /sys/class/net/ens2f0 /sys/class/net/ens2f1 /sys/class/net/lo /sys/class/net/rename6 /sys/class/net/rename7 /sys/class/net/virbr0 /sys/class/net/virbr0-nic
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:466 [[ default:br-rename6 = '' ]]
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:471 OVS_BRIDGES=default:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:474 bridge=br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:475 nic=rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:477 OVS_DPDK_PORT_MAPPINGS=rename6:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:522 MAPPINGS=rename6:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:527 nic=rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:528 bridge=br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:529 printf '%s in %s\n' rename6 br-rename6
2017-07-13 02:05:23.216 | rename6 in br-rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == eno1 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == eno2 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == ens2f0 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == ens2f1 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == lo ]]
2017-07-13 02:05:23.217 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 echo 0000:03:00.0#rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == rename6 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:533 NICS=0000:03:00.0#rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == rename7 ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == virbr0 ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == virbr0-nic ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:541 sudo sed 's/OVS_PCI_MAPPINGS=.*/OVS_PCI_MAPPINGS=0000:03:00.0#rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:542 sudo sed 's/OVS_BRIDGE_MAPPINGS=.*/OVS_BRIDGE_MAPPINGS=default:br-rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:543 sudo sed 's/OVS_DPDK_PORT_MAPPINGS=.*/OVS_DPDK_PORT_MAPPINGS=rename6:br-rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:24.088 | + local MAPPINGS=0000:03:00.0#rename6
2017-07-13 02:05:24.089 | ++ echo 0000:03:00.0#rename6
2017-07-13 02:11:00.764 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:44 [[ default:br-rename6 == '' ]]
2017-07-13 02:11:00.769 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:50 [[ default:br-rename6 != '' ]]
2017-07-13 02:11:00.773 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:51 iniset /etc/neutron/plugins/ml2/ml2_conf.ini ovs bridge_mappings default:br-rename6
```

Changed in networking-ovs-dpdk:
assignee: nobody → Tetsuro Nakamura (tetsuro0907)

Fix proposed to branch: master
Review: https://review.openstack.org/483863

Changed in networking-ovs-dpdk:
status: New → In Progress
Changed in networking-ovs-dpdk:
status: In Progress → Invalid
sean mooney (sean-k-mooney) wrote :

virtio-net interfaces need to to handeled in a different way then standard network interafaces as of dpdk 2.X before they changed to the date based releases.

looking at http://dpdk.org/doc/guides/nics/virtio.html#virtio-with-qemu-virtio-back-end
it may be safe to use uio_pci_generic as the kernel driver module but
i know there are issues with using igb_uio and vfio-pci.

vfio-pci need the kernel to be build with the soft iommu suport which i think is finally merged but
depending on the dpdk version you are using igb_uio may not be compatilbe with virto-net devices preventing transmition or recpetion of packets. i also know that there were issues binding the interfaces back to the kernel after being bound to igb_uio.

currently our ci whcih runs in vms uses e1000 nics instead as that works safly with all supported userspace drivers. i generally recommend useing e1000 nics if you plan on deploying ovs-dpdk in a vm for testing and development though virtio-net would give better perfromace if it works correctly with uio_pci_generic.

Changed in networking-ovs-dpdk:
status: Invalid → Triaged
importance: Undecided → Wishlist
Tetsuro Nakamura (tetsuro0907) wrote :

Mr.Sean

Thank you for the reply.
Then I try to use e1000 nics, since it's just for testing and development,

Change abandoned by Tetsuro Nakamura (<email address hidden>) on branch: master
Review: https://review.openstack.org/483863

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers