DPDK:3.2.x: After agent restart traffic to virtiovm is blackholed

[Vinod Nair]

On 3.2.12 DPDK compute on restarting vrouter-agent, traffic to the VM is blackholed.

Pkts arrive on the VIF but dont make it to VM. On restating/rebooting VM it starts working

oot@cs-scale-02:~# vifdump 3
vif0/3 PMD: tap240b8b45-f6
tcpdump: WARNING: mon3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on mon3, link-type EN10MB (Ethernet), capture size 65535 bytes
00:51:28.811996 IP 10.87.132.177 > 22.1.0.5: ICMP echo request, id 42831, seq 578, length 64
00:51:29.811979 IP 10.87.132.177 > 22.1.0.5: ICMP echo request, id 42831, seq 579, length 64
00:51:30.811995 IP 10.87.132.177 > 22.1.0.5: ICMP echo request, id 42831, seq 580, length 64
^C
3 packets captured
4 packets received by filter
0 packets dropped by kernel
vifdump: deleting vif 4348...
root@cs-scale-02:~# vif --get 3
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, Proxy=MAC Requests Proxied Always
       Mn=Mirror without Vlan Tag

vif0/3 PMD: tap240b8b45-f6
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:22.1.0.5
            Vrf:1 Flags:PL3L2D QOS:-1 Ref:23
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:0 bytes:0 errors:0
            TX packets:580 bytes:47544 errors:0
            TX port packets:580 errors:0 syscalls:580
            Drops:0

root@cs-scale-02:~# vif --get 3
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, Proxy=MAC Requests Proxied Always
       Mn=Mirror without Vlan Tag

vif0/3 PMD: tap240b8b45-f6
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:22.1.0.5
            Vrf:1 Flags:PL3L2D QOS:-1 Ref:23
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:0 bytes:0 errors:0
            TX packets:582 bytes:47740 errors:0
            TX port packets:582 errors:0 syscalls:582
            Drops:0

[Jeya ganesh babu J (jjeya)]

During a agent restart, the vif for the vm interface is deleted. This causes the stored feature list for the VM to be deleted as well. When agent comes up and the vif gets created, Qemu sends 0 for the feature list and this is causing communication issue. The Qemu feature storage code in dpdk vrouter needs to be revisited.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46604
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46632
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46633
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46634
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46604
Committed: http://github.com/Juniper/contrail-vrouter/commit/5991cac3a3180a638acb2f2ee6dc7b1a34e63bf6
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 5991cac3a3180a638acb2f2ee6dc7b1a34e63bf6
Author: Jeya ganesh babu J <email address hidden>
Date: Fri Sep 28 23:44:03 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted.
This causes the stored vhostuser feature set to
be deleted. Delete the stored feature only if
VM exits.

Change-Id: I06724440905b27b1087f294f8d9b6361144611f4

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46632
Committed: http://github.com/Juniper/contrail-vrouter/commit/a11e9fc4f9c40f402d20b5cd2d5e806a44f4ded0
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit a11e9fc4f9c40f402d20b5cd2d5e806a44f4ded0
Author: Jeya ganesh babu J <email address hidden>
Date: Fri Sep 28 23:44:03 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted.
This causes the stored vhostuser feature set to
be deleted. Delete the stored feature only if
VM exits.

Change-Id: I06724440905b27b1087f294f8d9b6361144611f4

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46633
Committed: http://github.com/Juniper/contrail-vrouter/commit/57c44b65dafb37f4b2632e42c60b715f2827cde2
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 57c44b65dafb37f4b2632e42c60b715f2827cde2
Author: Jeya ganesh babu J <email address hidden>
Date: Fri Sep 28 23:44:03 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted.
This causes the stored vhostuser feature set to
be deleted. Delete the stored feature only if
VM exits.

Change-Id: I06724440905b27b1087f294f8d9b6361144611f4

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46889
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46890
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46889
Committed: http://github.com/Juniper/contrail-vrouter/commit/4f26e8b4452f207559eba921c0f6a6d4b5de9642
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 4f26e8b4452f207559eba921c0f6a6d4b5de9642
Author: Jeya ganesh babu J <email address hidden>
Date: Wed Oct 10 18:15:11 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted. With server mode
configuration of vhostuser, we should not delete the socket
file in /var/run/vrouter.

Change-Id: I0050ea47e91151c7f3cb2f80191aa0c2bdddec70

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46634
Committed: http://github.com/Juniper/contrail-vrouter/commit/739d39bee0235d0ceadb372037797c43c40a8be1
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 739d39bee0235d0ceadb372037797c43c40a8be1
Author: Jeya ganesh babu J <email address hidden>
Date: Fri Sep 28 23:44:03 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted.
This causes the stored vhostuser feature set to
be deleted. Delete the stored feature only if
VM exits.

Change-Id: I06724440905b27b1087f294f8d9b6361144611f4

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46890
Committed: http://github.com/Juniper/contrail-vrouter/commit/fad5f9744e8db8533f863bed14dbf1b9d788a14d
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit fad5f9744e8db8533f863bed14dbf1b9d788a14d
Author: Jeya ganesh babu J <email address hidden>
Date: Wed Oct 10 18:15:11 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted. With server mode
configuration of vhostuser, we should not delete the socket
file in /var/run/vrouter.

Change-Id: I0050ea47e91151c7f3cb2f80191aa0c2bdddec70

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46921
Submitter: Jeya ganesh babu (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46921
Committed: http://github.com/Juniper/contrail-vrouter/commit/8d2e5da83b573343161ad6d45218f673d9d0dc17
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 8d2e5da83b573343161ad6d45218f673d9d0dc17
Author: Jeya ganesh babu J <email address hidden>
Date: Wed Oct 10 18:15:11 2018 -0700

VM traffic stop after agent restart.

Closes-bug: #1794934
During an agent restart, the vif gets deleted. With server mode
configuration of vhostuser, we should not delete the socket
file in /var/run/vrouter.

Change-Id: I0050ea47e91151c7f3cb2f80191aa0c2bdddec70
(cherry picked from commit 4f26e8b4452f207559eba921c0f6a6d4b5de9642)