OpenvSwitch with DPDK brings all VirtIO NICs down, software reboot also doesn't work.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
openvswitch (Ubuntu) |
Fix Released
|
Low
|
Unassigned |
Bug Description
Guys,
Soon as we start OpenvSwitch with DPDK inside of a KVM guest, it brings down all the VirtIO NICs! Including non-DPDK NIC VirtIO devices!
The problem is easy to reproduce, I'm seeing it on two different hosts, a Dell server and a Macbook Pro, both running Xenial on host and on VM.
Also, if DPDK Openvswitch options are enabled on /etc/default/
Steps to reproduce:
1- At the host, install Ubuntu KVM by running:
sudo apt install ubuntu-virt
2- Create two extra Libvirt networks, called "subscriber" and "internet", list this:
tmartins@blade:~$ virsh net-list
Name State Autostart Persistent
-------
default active yes yes
internet active yes yes
subscriber active yes yes
3- Create a Xenial VM with 3 vNIC (Virtio), wire it against those 3 above netorks.
4- Inside of the Xenial VM, you might see the PCI IDs like this:
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:09.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0a.0 Ethernet controller: Red Hat, Inc Virtio network device
5- Install OpenvSwitch with DPDK:
sudo apt install openvswitch-
6- Configure DPDK:
sudo vi /etc/dpdk/dpdk.conf # Enable NR_2M_PAGES=64
7- Select the PCI devices that you want to bind to DPDK drivers:
sudo vi /etc/dpdk/
pci 0000:00:09.0 uio_pci_generic
pci 0000:00:0a.0 uio_pci_generic
8- Reboot it
NOTE: After rebooting it, you'll see by running "dpdk_nic_bind --status", that the latest two VirtIO NIC cards are using DPDK-compatible drivers. So far, so good...
9- Configure OpenvSwitch to use DPDK
run:
update-
sudo vi /etc/default/
Uncomment the latest line, like this:
DPDK_OPTS='--dpdk -c 0x1 -n 4'
10- Hit the bug!
Run:
service openvswitch-switch stop
service openvswitch-switch start # BOOM!!!
From this point, there are two problems:
1- All VirtIO NIC cards loses connectivity, not only the DPDK ones! But also, vNICs with the regular Linux VirtIO drivers!
2- Soft reboot does not work.
That's it! What am I missing here?
Workaround:
1- Disable OpenvSwitch DPDK options inside of /etc/default/
I made a video about this! It is on Youtube:
I really appreciate any help!
Cheers!
Thiago
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: openvswitch-
ProcVersionSign
Uname: Linux 4.4.0-18-generic x86_64
ApportVersion: 2.20.1-0ubuntu1
Architecture: amd64
Date: Sun Apr 10 18:25:34 2016
InstallationDate: Installed on 2016-02-01 (69 days ago)
InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160111)
SourcePackage: openvswitch
UpgradeStatus: No upgrade log present (probably fresh install)
Guys,
I can confirm that this problem does exists on top of Trusty KVM Guest with Linux 4.4 (same Xenial kernel), but with DPDK 2.0 and OpenvSwitch 2.4 from Liberty Cloud Archive, it works!!
At least, the Trusty guest doesn't lose connectivity and soft reboot works too! Precisely where Xenial fails...
So, something is wrong with OpenvSwitch 2.5 when with DPDK 2.2 (Xenial Combo).
Happy hacking!
Thanks!
Thiago