traffic lost during vrouter-agent restart

[Foucault de Bonneval]

Release : On a 1.06
Component : vrouter-agent

 when I restart vrouter-agent on a compute node traffic is lost in both ways during a few seconds like 5 for a plateform with 1 VM to 10 for a plateform with 15-20 VMs

From the outside :
^^:~$ ping 84.39.63.170
PING 84.39.63.170 (84.39.63.170): 56 data bytes
64 bytes from 84.39.63.170: icmp_seq=0 ttl=57 time=2.834 ms
64 bytes from 84.39.63.170: icmp_seq=1 ttl=57 time=1.557 ms
64 bytes from 84.39.63.170: icmp_seq=2 ttl=57 time=1.604 ms
64 bytes from 84.39.63.170: icmp_seq=3 ttl=57 time=1.666 ms
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
64 bytes from 84.39.63.170: icmp_seq=8 ttl=57 time=154.615 ms
64 bytes from 84.39.63.170: icmp_seq=9 ttl=57 time=1.584 ms
64 bytes from 84.39.63.170: icmp_seq=10 ttl=57 time=1.600 ms
64 bytes from 84.39.63.170: icmp_seq=11 ttl=57 time=4.501 ms
^C
--- 84.39.63.170 ping statistics ---
12 packets transmitted, 8 packets received, 33.3% packet loss
round-trip min/avg/max/stddev = 1.557/21.245/154.615/50.418 ms

From the inside :
cloud@vrouter-agent-test:~$ ping www.opencontrail.org
PING www.opencontrail.org (192.237.193.210) 56(84) bytes of data.
[...]
64 bytes from 192.237.193.210 (192.237.193.210): icmp_seq=14 ttl=53 time=123 ms
64 bytes from 192.237.193.210 (192.237.193.210): icmp_seq=19 ttl=53 time=123 ms
64 bytes from 192.237.193.210 (192.237.193.210): icmp_seq=20 ttl=53 time=132 ms
64 bytes from 192.237.193.210 (192.237.193.210): icmp_seq=21 ttl=53 time=123 ms
--- www.opencontrail.org ping statistics ---
21 packets transmitted, 17 received, 19% packet loss, time 20036ms
rtt min/avg/max/mdev = 123.042/128.523/132.798/4.377 ms

[Prabhjot Singh Sethi (prabhjot)]

As per current contrail-vrouter-agent design on restart it resets vrouter(kernel dataplane module) and starts reprogramming as and when it starts receiving config and routes, to keep vrouter and vrouter-agent always in Sync.

because of this on vrouter-agent restart, vrouter will not have forwarding information for a short while and will result in drops/traffic loss.

To Achieve graceful restart of vrouter agent we need to have a mechanism to read back forwarding plane information from vrouter and then should be able to audit and cleanup the stale information.