DPDK: system runs out of mbufs after flipping protocol in the traffic

Bug #1516873 reported by Vinod Nair
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
Critical
Vinod Nair
Trunk
Fix Committed
Critical
rch

Bug Description

On a VM initially had 40K tcp flows, after slipping the protocol to UDP with the same 40K flows, the system runs out of mbufs
The only way to recover is by restarting the vrouter

vif0/0 PCI: 0:0:0.0 (Speed 10000, Duplex 1)
            Type:Physical HWaddr:90:e2:ba:4c:66:e8 IPaddr:0
            Vrf:0 Flags:L3L2Vp MTU:1514 Ref:27
            RX device packets:1422004090 bytes:130824498245 errors:530407970 no mbufs:8086096960
            RX port packets:1422004090 errors:0
            RX queue packets:1420899986 errors:1066401
            RX queue to lcore 10 errors:152382
            RX queue to lcore 11 errors:157667
            RX queue to lcore 12 errors:150550
            RX queue to lcore 13 errors:148048
            RX queue to lcore 14 errors:151420
            RX queue to lcore 15 errors:152326
            RX queue to lcore 16 errors:154008
            RX packets:1420937689 bytes:125042638597 errors:0
            TX packets:178543 bytes:251374160 errors:0
            TX port packets:178543 errors:0
            TX device packets:178543 bytes:251374160 errors:0

root@cs-scale-8:/var/log/contrail# vif --get 5
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload

vif0/5 PMD: tap647e84a5-d5
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:0
            Vrf:1 Flags:PL3L2D MTU:9160 Ref:17
            RX port packets:28 errors:0 syscalls:1
            RX queue packets:28 errors:0
            RX packets:28 bytes:2492 errors:0
            TX packets:1393657545 bytes:83619428524 errors:0
            TX port packets:1393493535 errors:164386 syscalls:12

CPU core mask is 0xff

logs in http://cmbu-sv02.englab.juniper.net/pxe/Standard/vin/dpdk/log2/

Tags: dpdk
Vinod Nair (vinodnair)
summary: - DPDK: system runs out of mbufs after flipping flows
+ DPDK: system runs out of mbufs after flipping protocol in the traffic
description: updated
Revision history for this message
rch (rch-c) wrote :

I can not reproduce the bug, could you write step to reproduce it (and add version of *.deb).

Revision history for this message
Vinod Nair (vinodnair) wrote :

I am sending traffic from a traffic gen ( IXIA) with incremental source port (40K and fixed dest port @ 1M PPS
Initially send with with protocol as tcp . stop the traffic and send with protocol as UDP

The build is the version Given to att built by raja

Jeba Paulaiyan (jebap)
information type: Proprietary → Public
Revision history for this message
Vinod Nair (vinodnair) wrote :

This issue is no longer seen in 3.0.2 images, fixed by some other commit

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.