Dual MX: VM To BMS ping is getting discarded

[chhandak]

In Dual MX with ECMP scenario, ping from VM to BMS is failing.

Ping response is coming till compute node back and then getting discarded. Discarded because of flow action drop.

root@nodei5:~# dropstats | grep 'Flow Action Drop'
Flow Action Drop 30618
root@nodei5:~# dropstats | grep 'Flow Action Drop'
Flow Action Drop 30619
root@nodei5:~# dropstats | grep 'Flow Action Drop'
Flow Action Drop 30620

root@nodei5:~# tcpdump -ni eth3 udp port 4789
tcpdump: WARNING: eth3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes
21:02:25.394338 IP 34.34.33.33.13024 > 192.168.194.5.4789: UDP, length 68
21:02:25.394459 IP 192.168.194.5.56168 > 34.34.33.33.4789: UDP, length 68
21:02:25.418199 IP 192.168.194.5.49282 > 34.34.33.33.4789: UDP, length 106
21:02:25.418405 IP 34.34.33.33.40917 > 192.168.194.5.4789: UDP, length 106
21:02:26.426638 IP 192.168.194.5.49282 > 34.34.33.33.4789: UDP, length 106
21:02:26.426776 IP 34.34.33.33.40917 > 192.168.194.5.4789: UDP, length 106
21:02:27.434092 IP 192.168.194.5.49282 > 34.34.33.33.4789: UDP, length 106
21:02:27.434313 IP 34.34.33.33.40917 > 192.168.194.5.4789: UDP, length 106

root@nodei5:~# flow -l
Flow table(size 34078720, entries 532480)

Entries: Created 60963 Added 30492 Processed 60963
(Created Flows/CPU: 399 843 3925 333 100 57 67 47 21975 790 588 247 305 217 224 25975 0 4712 2 1 0 0 1 0 3 16 7 0 1 1 0 127)(oflows 0)

Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
 Other:K(nh)=Key_Nexthop, S(nh)=RPF_Nexthop

 Index Source:Port Destination:Port Proto(V)
-------------------------------------------------------------------------
  3352 1.1.1.7:54273 1.1.1.2:53 17 (1)
        (K(nh):52, Action:F, S(nh):52, Statistics:1/100 UdpSrcPort 62313)

 89588 1.1.1.7:46039 1.1.1.2:53 17 (1)
        (K(nh):52, Action:F, S(nh):52, Statistics:1/100 UdpSrcPort 53000)

139568 1.1.1.7:3463 1.1.1.3:0 1 (1)
        (K(nh):52, Action:D(RevFlowChng), S(nh):52, Statistics:1/98 UdpSrcPort 49282) >>> Action drop because of RevFlow Change

170156 1.1.1.3:3463 1.1.1.7:0 1 (1)
        (K(nh):13, Action:D(RevFlowChng), S(nh):22, Statistics:1/98 UdpSrcPort 58012)

233136 1.1.1.2:53 1.1.1.7:54273 17 (1)
        (K(nh):52, Action:F, S(nh):7, Statistics:0/0 UdpSrcPort 62387)

320164 1.1.1.2:53 1.1.1.7:46039 17 (1)
        (K(nh):52, Action:F, S(nh):7, Statistics:0/0 UdpSrcPort 62230)

459120 1.1.1.3:3463 1.1.1.7:0 1 (1)
        (K(nh):52, Action:F, S(nh):22, Statistics:0/0 UdpSrcPort 58012)

Ping from BMS to VM is Fine
root@nodeh8:~# ping 1.1.1.7
PING 1.1.1.7 (1.1.1.7) 56(84) bytes of data.
64 bytes from 1.1.1.7: icmp_seq=1 ttl=64 time=2.11 ms
64 bytes from 1.1.1.7: icmp_seq=2 ttl=64 time=0.720 ms
64 bytes from 1.1.1.7: icmp_seq=3 ttl=64 time=0.637 ms

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/11909
Submitter: Divakar Dharanalakota (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/11914
Submitter: Divakar Dharanalakota (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/11909
Committed: http://github.org/Juniper/contrail-vrouter/commit/a264f4244b18ee9593628833d7eda165d766cfe4
Submitter: Zuul
Branch: R2.20

commit a264f4244b18ee9593628833d7eda165d766cfe4
Author: Divakar <email address hidden>
Date: Mon Jun 22 10:30:10 2015 +0530

Disabling Source lookup for Vxlan packets

When the Vxlan packet from TOR are reaching a compute node,
before processing them in Vrf translate nexthop, we verify,
in nh_output(), whether flow processing needs to happen or not. IF
policy bit is not enabled in the nexthop, we do inner source IP lookup
to identify if it is ECMP source. If so, we force the flow processing.
Incase of control node peering with two MX, the BMS subnet would be
advertised by both MX leading to ECMP source . This is forcing the vxlan
tagged packets to be trapped to AGent with nh_id as VRF translate
nexthop. This leads to Agent marking the flow as short flow as Agent
already has set this flow with l3_nhid.
To avoid this scenario, the source lookup is done only for non vxlan
packets. This check of verifyin whether this is a vxlan packet or not
enabled under a proc entry vxlan_ecmp_no_rpf and by default this is
turned on, which means we dont do RPF check for VXlan packets. If
someone wants to do RPF check even for Vxlan tagged packets, they can
set this to 0.

Change-Id: I4446a0b8171951f206cc13e12855be506b73e53d
closes-bug: #1467273

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/11914
Committed: http://github.org/Juniper/contrail-vrouter/commit/071c8d75718050e7037f44e661000fbd10c81557
Submitter: Zuul
Branch: master

commit 071c8d75718050e7037f44e661000fbd10c81557
Author: Divakar <email address hidden>
Date: Mon Jun 22 12:26:36 2015 +0530

Disabling Source lookup for Vxlan packets

When the Vxlan packet from TOR are reaching a compute node,
before processing them in Vrf translate nexthop, we verify,
in nh_output(), whether flow processing needs to happen or not. IF
policy bit is not enabled in the nexthop, we do inner source IP lookup
to identify if it is ECMP source. If so, we force the flow processing.
Incase of control node peering with two MX, the BMS subnet would be
advertised by both MX leading to ECMP source . This is forcing the vxlan
tagged packets to be trapped to AGent with nh_id as VRF translate
nexthop. This leads to Agent marking the flow as short flow as Agent
already has set this flow with l3_nhid.

To avoid this scenario, the source lookup is done only for INET packets

Change-Id: Ic63c05ade2c7ee3757003d6cace8595eb015dc7a
closes-bug: #1467273