Juniper Openstack

DPDK vrouter doesnot come up with vfio-pci

Bug #1787303 reported by Vinod Nair
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
Critical
Jeya ganesh babu J
Juniper Openstack r5.0.2
Trunk
Fix Committed
Critical
Jeya ganesh babu J
Juniper Openstack r5.1.0

Bug Description

DPDK vrouter with uio driver as vfio-pci doesn't come up.uio_pci_generic is ok

The issue seems to be with dpdk 17.11.3

Build:
ocata-5.0-214
root@cs-scale-02 vin]# 2018-08-15 19:22:34,281 DPCORE: vrouter soft reset start
2018-08-15 19:22:34,281 VROUTER: Deleting vif 0 eth device
2018-08-15 19:22:34,281 VROUTER: releasing lcore 8 TX queue 0
2018-08-15 19:22:34,281 VROUTER: releasing lcore 9 TX queue 0
2018-08-15 19:22:34,281 VROUTER: releasing lcore 10 RX queue
2018-08-15 19:22:34,281 VROUTER: releasing lcore 10 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 11 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 11 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 12 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 12 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 13 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 13 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 14 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 14 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 15 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 15 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 16 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 16 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 17 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 17 TX queue 0
2018-08-15 19:22:34,282 VROUTER: Deleting vif 1 device
2018-08-15 19:22:34,282 VROUTER: releasing lcore 8 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 9 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 10 RX queue
2018-08-15 19:22:34,282 VROUTER: releasing lcore 10 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 11 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 12 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 13 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 14 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 15 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 16 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing lcore 17 TX queue 0
2018-08-15 19:22:34,282 VROUTER: releasing vif 1 TAP device
2018-08-15 19:22:34,335 DPCORE: vrouter soft reset done (0)
2018-08-15 19:22:34,341 VROUTER: Notification received for vhost0
2018-08-15 19:22:34,341 VROUTER: vr_dpdk_handle_vhost0_notification error: NULL ethdev
2018-08-15 19:22:34,375 VROUTER: Adding vif 0 (gen. 3) eth device 2 PCI 0000:00:00.0 MAC 92:77:06:8e:7a:6e (vif MAC 92:77:06:8e:7a:6e)
2018-08-15 19:22:34,375 VROUTER: Using 64 TX queues, 8 RX queues
2018-08-15 19:22:34,375 PMD: __eth_bond_slave_add_lock_free(273) - Slave device is already a slave of a bonded device
2018-08-15 19:22:34,375 EAL: Failed to add port 0 as slave to bonded device eth_bond_bond0.141
2018-08-15 19:22:34,375 PMD: __eth_bond_slave_add_lock_free(273) - Slave device is already a slave of a bonded device
2018-08-15 19:22:34,375 EAL: Failed to add port 1 as slave to bonded device eth_bond_bond0.141
2018-08-15 19:22:34,375 VROUTER: bond eth device 2 configured MAC 92:77:06:8e:7a:6e
2018-08-15 19:22:34,375 VROUTER: bond member eth device 0 PCI 0000:81:00.0 MAC 68:05:ca:31:07:e8
2018-08-15 19:22:34,375 VROUTER: bond member eth device 0: unable to add MAC addresses
2018-08-15 19:22:34,375 VROUTER: bond member eth device 1 PCI 0000:81:00.1 MAC 68:05:ca:31:07:ea
2018-08-15 19:22:34,375 VROUTER: bond member eth device 1: unable to add MAC addresses
2018-08-15 19:22:34,375 VROUTER: setup 8 RSS queue(s) and 0 filtering queue(s)
2018-08-15 19:22:35,063 VROUTER: lcore 10 TX to HW queue 0
2018-08-15 19:22:35,063 VROUTER: lcore 11 TX to HW queue 1
2018-08-15 19:22:35,063 VROUTER: lcore 12 TX to HW queue 2
2018-08-15 19:22:35,063 VROUTER: lcore 13 TX to HW queue 3
2018-08-15 19:22:35,063 VROUTER: lcore 14 TX to HW queue 4
2018-08-15 19:22:35,063 VROUTER: lcore 15 TX to HW queue 5
2018-08-15 19:22:35,063 VROUTER: lcore 16 TX to HW queue 6
2018-08-15 19:22:35,063 VROUTER: lcore 17 TX to HW queue 7
2018-08-15 19:22:35,063 VROUTER: lcore 8 TX to HW queue 8
2018-08-15 19:22:35,063 VROUTER: lcore 9 TX to HW queue 9
2018-08-15 19:22:35,063 VROUTER: lcore 10 RX from HW queue 0
2018-08-15 19:22:35,063 VROUTER: lcore 11 RX from HW queue 1
2018-08-15 19:22:35,063 VROUTER: lcore 12 RX from HW queue 2
2018-08-15 19:22:35,063 VROUTER: lcore 13 RX from HW queue 3
2018-08-15 19:22:35,063 VROUTER: lcore 14 RX from HW queue 4
2018-08-15 19:22:35,063 VROUTER: lcore 15 RX from HW queue 5
2018-08-15 19:22:35,063 VROUTER: lcore 16 RX from HW queue 6
2018-08-15 19:22:35,063 VROUTER: lcore 17 RX from HW queue 7
2018-08-15 19:22:35,063 VROUTER: Adding vif 1 (gen. 4) device vhost0 at eth device 2 MAC 92:77:06:8e:7a:6e (vif MAC 92:77:06:8e:7a:6e)
2018-08-15 19:22:35,063 VROUTER: using bond slave eth device 0 MAC 68:05:ca:31:07:e8
2018-08-15 19:22:35,063 VROUTER: KNI is not available
2018-08-15 19:22:35,063 VROUTER: creating TAP device vhost0
2018-08-15 19:22:35,063 VROUTER: TAP device vhost0 already exists
2018-08-15 19:22:35,063 VROUTER: lcore 10 TX to HW queue 0
2018-08-15 19:22:35,063 VROUTER: lcore 11 TX to HW queue 1
2018-08-15 19:22:35,063 VROUTER: lcore 12 TX to HW queue 2
2018-08-15 19:22:35,063 VROUTER: lcore 13 TX to HW queue 3
2018-08-15 19:22:35,063 VROUTER: lcore 14 TX to HW queue 4
2018-08-15 19:22:35,063 VROUTER: lcore 15 TX to HW queue 5
2018-08-15 19:22:35,063 VROUTER: lcore 16 TX to HW queue 6
2018-08-15 19:22:35,063 VROUTER: lcore 17 TX to HW queue 7
2018-08-15 19:22:35,063 VROUTER: lcore 8 TX to HW queue 8
2018-08-15 19:22:35,063 VROUTER: lcore 9 TX to HW queue 9
2018-08-15 19:22:35,063 VROUTER: lcore 10 RX from HW queue 0
2018-08-15 19:22:35,063 VROUTER: Adding vif 2 (gen. 5) packet device unix
2018-08-15 19:22:35,063 USOCK: usock_alloc[7f8d8f99b700]: new socket FD 77
2018-08-15 19:22:35,063 USOCK: usock_alloc[7f8d8f99b700]: setting socket FD 77 send buff size.
Buffer size set to 18320000 (requested 9216000)

Workaround is to use dpdk 17.11 Binary

See original description
Vinod Nair (vinodnair)
description: updated
Vinod Nair (vinodnair)
description: updated
Vinod Nair (vinodnair)
information type: Proprietary → Public
Revision history for this message
Vinod Nair (vinodnair) wrote :
Jeba Paulaiyan (jebap)
tags: added: releasenote
Revision history for this message
Jeba Paulaiyan (jebap) wrote :

Note:

The DPDK 17.11.3 binary shipped with 5.0.1 release works with NICs using uio_pci_generic driver. For NICs using vfio-pci driver, change the DPDK version to 17.11 using below steps:

1. Copy the DPDK 17.11 binary (from LP 1787303) to /var/tmp of the vrouter DPDK container
2. Login to the container and move the binary to /usr/bin/contrail-vrouter-dpdk
3. Restart the DPDK vrouter container

Revision history for this message
Vinod Nair (vinodnair) wrote :

The issue is resolved by updating to latest NIC firmware from intel. DPDK 17.11.3 works fine with the below firmware.

ethtool -i enp129s0f0
driver: i40e
version: 2.1.14-k
firmware-version: 6.01 0x80003493 255.65535.255
expansion-rom-version:
bus-info: 0000:81:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Revision history for this message
Sachchidanand Vaidya (vaidyasd) wrote :

Fix is to upgrade firmware driver.

The issue is resolved by updating to latest NIC firmware from intel. DPDK 17.11.3 works fine with the below firmware.

ethtool -i enp129s0f0
driver: i40e
version: 2.1.14-k
firmware-version: 6.01 0x80003493 255.65535.255
expansion-rom-version:
bus-info: 0000:81:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Revision history for this message
Sachchidanand Vaidya (vaidyasd) wrote :

FIx is to upgrade Firware for NIc

The issue is resolved by updating to latest NIC firmware from intel. DPDK 17.11.3 works fine with the below firmware.

ethtool -i enp129s0f0
driver: i40e
version: 2.1.14-k
firmware-version: 6.01 0x80003493 255.65535.255
expansion-rom-version:
bus-info: 0000:81:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.