networking-ovs-dpdk

Ovs-dpdk doesn't grab virtio NIC port via devstack

Bug #1704279 reported by Tetsuro Nakamura on 2017-07-14

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	networking-ovs-dpdk	Triaged	Wishlist	Tetsuro Nakamura

Bug Description

Description
===========

When you map a virtio backend NIC port to ovs-dpdk with devstack, ovs-dpdk doesn't grab the NIC port, and you get no error.

Steps to reproduce
==================

1. create devstack to map a virtio-backend port to ovs-dpdk
The local.conf includes like the following line,
```
OVS_BRIDGE_MAPPINGS=default:br-ens4
```
where ens4 is virtio backend.
```
$ sudo ethtool -i ens4
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:00:04.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
```

2. run stack.sh to see that after all ovs is't connected the NIC port
```
$ sudo ovs-vsctl show
e3f789a9-6061-4ec7-a592-06cbc7be2160
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge "br-ens4"
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "phy-br-ens4"
            Interface "phy-br-ens4"
                type: patch
                options: {peer="int-br-ens4"}
        Port "br-ens4"
            Interface "br-ens4"
                type: internal
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "int-br-ens4"
            Interface "int-br-ens4"
                type: patch
                options: {peer="phy-br-ens4"}
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
```

Expected result
===============

Devstack connects ovs to the specified port appropriately.

Environment
===========

I'm runnning devstack on ubuntu16.04 built using KVM on ubuntu16.04

Logs & Configs
==============
stack.sh.log is below
```
$ cat /opt/stack/logs/stack.sh.log | grep ens4
2017-07-13 09:43:00.089 | ++/opt/stack/networking-ovs-dpdk/devstack/settings:source:44 OVS_BRIDGE_MAPPINGS=default:br-ens4
2017-07-13 09:47:47.760 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:464 ls -al /sys/class/net/ens3 /sys/class/net/ens4 /sys/class/net/lo
2017-07-13 09:47:47.785 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:466 [[ default:br-ens4 = '' ]]
2017-07-13 09:47:47.805 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:471 OVS_BRIDGES=default:br-ens4
2017-07-13 09:47:47.839 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:474 bridge=br-ens4
2017-07-13 09:47:47.854 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:475 nic=ens4
2017-07-13 09:47:47.879 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:477 OVS_DPDK_PORT_MAPPINGS=ens4:br-ens4
2017-07-13 09:47:47.930 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:522 MAPPINGS=ens4:br-ens4
2017-07-13 09:47:47.980 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:527 nic=ens4
2017-07-13 09:47:47.991 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:528 bridge=br-ens4
2017-07-13 09:47:48.002 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:529 printf '%s in %s\n' ens4 br-ens4
2017-07-13 09:47:48.002 | ens4 in br-ens4
2017-07-13 09:47:48.046 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == ens3 ]]
2017-07-13 09:47:48.072 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 echo virtio1#ens4
2017-07-13 09:47:48.084 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == ens4 ]]
2017-07-13 09:47:48.109 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:533 NICS=virtio1#ens4
2017-07-13 09:47:48.151 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ ens4 == lo ]]
2017-07-13 09:47:48.247 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:541 sudo sed 's/OVS_PCI_MAPPINGS=.*/OVS_PCI_MAPPINGS=virtio1#ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:48.275 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:542 sudo sed 's/OVS_BRIDGE_MAPPINGS=.*/OVS_BRIDGE_MAPPINGS=default:br-ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:48.302 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:543 sudo sed 's/OVS_DPDK_PORT_MAPPINGS=.*/OVS_DPDK_PORT_MAPPINGS=ens4:br-ens4/' -i /etc/default/ovs-dpdk
2017-07-13 09:47:49.778 | + local MAPPINGS=virtio1#ens4
2017-07-13 09:47:49.779 | ++ echo virtio1#ens4
2017-07-13 09:48:28.128 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:44 [[ default:br-ens4 == '' ]]
2017-07-13 09:48:28.137 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:50 [[ default:br-ens4 != '' ]]
2017-07-13 09:48:28.147 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:51 iniset /etc/neutron/plugins/ml2/ml2_conf.ini ovs bridge_mappings default:br-ens4
```

It is expected to be like below, where rename6 is a physical NIC port.
```
$ cat /opt/stack/logs/stack.sh.log | grep rename6
2017-07-13 01:59:58.066 | ++/opt/stack/networking-ovs-dpdk/devstack/settings:source:44 OVS_BRIDGE_MAPPINGS=default:br-rename6
2017-07-13 02:05:23.216 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:464 ls -al /sys/class/net/eno1 /sys/class/net/eno2 /sys/class/net/ens2f0 /sys/class/net/ens2f1 /sys/class/net/lo /sys/class/net/rename6 /sys/class/net/rename7 /sys/class/net/virbr0 /sys/class/net/virbr0-nic
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:466 [[ default:br-rename6 = '' ]]
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:471 OVS_BRIDGES=default:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:474 bridge=br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:475 nic=rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:477 OVS_DPDK_PORT_MAPPINGS=rename6:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:522 MAPPINGS=rename6:br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:527 nic=rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:528 bridge=br-rename6
2017-07-13 02:05:23.216 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:529 printf '%s in %s\n' rename6 br-rename6
2017-07-13 02:05:23.216 | rename6 in br-rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == eno1 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == eno2 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == ens2f0 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == ens2f1 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == lo ]]
2017-07-13 02:05:23.217 | +++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 echo 0000:03:00.0#rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == rename6 ]]
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:533 NICS=0000:03:00.0#rename6
2017-07-13 02:05:23.217 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == rename7 ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == virbr0 ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:531 [[ rename6 == virbr0-nic ]]
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:541 sudo sed 's/OVS_PCI_MAPPINGS=.*/OVS_PCI_MAPPINGS=0000:03:00.0#rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:542 sudo sed 's/OVS_BRIDGE_MAPPINGS=.*/OVS_BRIDGE_MAPPINGS=default:br-rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:23.218 | ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:ovs_dpdk_write_conf:543 sudo sed 's/OVS_DPDK_PORT_MAPPINGS=.*/OVS_DPDK_PORT_MAPPINGS=rename6:br-rename6/' -i /etc/default/ovs-dpdk
2017-07-13 02:05:24.088 | + local MAPPINGS=0000:03:00.0#rename6
2017-07-13 02:05:24.089 | ++ echo 0000:03:00.0#rename6
2017-07-13 02:11:00.764 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:44 [[ default:br-rename6 == '' ]]
2017-07-13 02:11:00.769 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:50 [[ default:br-rename6 != '' ]]
2017-07-13 02:11:00.773 | +lib/neutron_plugins/openvswitch_agent:neutron_plugin_configure_plugin_agent:51 iniset /etc/neutron/plugins/ml2/ml2_conf.ini ovs bridge_mappings default:br-rename6
```

Tetsuro Nakamura (tetsuro0907) on 2017-07-14

Changed in networking-ovs-dpdk:
assignee:	nobody → Tetsuro Nakamura (tetsuro0907)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-07-14: Fix proposed to networking-ovs-dpdk (master)

Fix proposed to branch: master
Review: https://review.openstack.org/483863

Changed in networking-ovs-dpdk:
status:	New → In Progress

sean mooney (sean-k-mooney) on 2017-07-23

Changed in networking-ovs-dpdk:
status:	In Progress → Invalid

Revision history for this message

sean mooney (sean-k-mooney) wrote on 2017-07-23:

virtio-net interfaces need to to handeled in a different way then standard network interafaces as of dpdk 2.X before they changed to the date based releases.

looking at http://dpdk.org/doc/guides/nics/virtio.html#virtio-with-qemu-virtio-back-end
it may be safe to use uio_pci_generic as the kernel driver module but
i know there are issues with using igb_uio and vfio-pci.

vfio-pci need the kernel to be build with the soft iommu suport which i think is finally merged but
depending on the dpdk version you are using igb_uio may not be compatilbe with virto-net devices preventing transmition or recpetion of packets. i also know that there were issues binding the interfaces back to the kernel after being bound to igb_uio.

currently our ci whcih runs in vms uses e1000 nics instead as that works safly with all supported userspace drivers. i generally recommend useing e1000 nics if you plan on deploying ovs-dpdk in a vm for testing and development though virtio-net would give better perfromace if it works correctly with uio_pci_generic.

Changed in networking-ovs-dpdk:
status:	Invalid → Triaged
importance:	Undecided → Wishlist

Revision history for this message

Tetsuro Nakamura (tetsuro0907) wrote on 2017-07-25:

Mr.Sean

Thank you for the reply.
Then I try to use e1000 nics, since it's just for testing and development,

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-03: Change abandoned on networking-ovs-dpdk (master)

Change abandoned by Tetsuro Nakamura (<email address hidden>) on branch: master
Review: https://review.openstack.org/483863

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.