Gateway_less_Fwd: ECMP is not working when IP Fabric is configured as provider network

[Chandra Sekhar Reddy Mallam]

ECMP is not working when IP Fabric is configured as provider network

R4.1.0.0 Build 23 Ubuntu 14.04 Mitaka

Steps
———
Create a VN (say left_vn: 10.10.10.0/24)and configure provider network as IP Fabric network
Launch 3 VMs across compute nodes in left_vn
Now, configure AAP (say 10.10.10.100/32) on 2 VMIs with active-active mode
Now, AAP route shows as ECMP properly on left_vn.
But, whenthe AAP route leaked to default routing instance due to IP-Fabric is configured as provider network, it is not ECMP.

Please see the log below:

root@nodek11:~# vif --list
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
       Proxy=MAC Requests Proxied Always, Er=Etree Root

vif0/0 OS: em1 (Speed 1000, Duplex 1)
            Type:Physical HWaddr:0c:c4:7a:32:0a:88 IPaddr:0.0.0.0
            Vrf:0 Flags:L3L2VpEr QOS:-1 Ref:6
            RX packets:27157 bytes:2857641 errors:2
            TX packets:6929 bytes:4887312 errors:0
            Drops:2

vif0/1 OS: vhost0
            Type:Host HWaddr:0c:c4:7a:32:0a:88 IPaddr:10.204.216.231
            Vrf:0 Flags:PL3DEr QOS:-1 Ref:7
            RX packets:6489 bytes:4744582 errors:0
            TX packets:27325 bytes:2906745 errors:0
            Drops:1

vif0/2 OS: pkt0
            Type:Agent HWaddr:00:00:5e:00:01:00 IPaddr:0.0.0.0
            Vrf:65535 Flags:L3Er QOS:-1 Ref:3
            RX packets:1975 bytes:241450 errors:0
            TX packets:12668 bytes:1364863 errors:0
            Drops:0

vif0/3 OS: tap7fe679ef-96
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:20.20.20.3
            Vrf:3 Flags:PL3L2DEr QOS:-1 Ref:5
            RX packets:1163 bytes:106778 errors:0
            TX packets:697 bytes:65390 errors:0
            Drops:1408

vif0/4 OS: tap0346accc-11
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.10.10.3 <=== AAP is configured on this VMI and also on one more VMI which is in another compute
            Vrf:0 Flags:PL3L2DProxyEr QOS:-1 Ref:5
            RX packets:1161 bytes:105964 errors:0
            TX packets:1191 bytes:94846 errors:0
            Drops:468

vif0/5 OS: tapb0cdc481-1a
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.10.10.4
            Vrf:0 Flags:PL3L2DProxyEr QOS:-1 Ref:5
            RX packets:2901 bytes:355584 errors:0
            TX packets:2465 bytes:207514 errors:0
            Drops:342

vif0/4350 OS: pkt3
            Type:Stats HWaddr:00:00:00:00:00:00 IPaddr:0.0.0.0
            Vrf:65535 Flags:L3L2 QOS:0 Ref:1
            RX packets:0 bytes:0 errors:0
            TX packets:0 bytes:0 errors:0
            Drops:0

vif0/4351 OS: pkt1
            Type:Stats HWaddr:00:00:00:00:00:00 IPaddr:0.0.0.0
            Vrf:65535 Flags:L3L2 QOS:0 Ref:1
            RX packets:981 bytes:82404 errors:0
            TX packets:981 bytes:82404 errors:0
            Drops:0

root@nodek11:~# rt --dump 2 | grep 10.10.10.100 <=== ECMP on AAP on left_vn VRF
10.10.10.100/32 32 P - 33 -
root@nodek11:~# nh --get 33
Id:33 Type:Composite Fmly: AF_INET Rid:0 Ref_cnt:2 Vrf:2
              Flags:Valid, Policy, Ecmp, Etree Root,
              Valid Hash Key Parameters: Proto,SrcIP,SrcPort,DstIp,DstPort
              Sub NH(label): 31(21) 22(26)

Id:31 Type:Tunnel Fmly: AF_INET Rid:0 Ref_cnt:11 Vrf:0
              Flags:Valid, MPLSoUDP, Etree Root,
              Oif:0 Len:14 Data:00 25 90 93 d2 44 0c c4 7a 32 0a 88 08 00
              Sip:10.204.216.231 Dip:10.204.216.34

Id:22 Type:Encap Fmly: AF_INET Rid:0 Ref_cnt:4 Vrf:0
              Flags:Valid, Etree Root,
              EncapFmly:0806 Oif:4 Len:14
              Encap Data: 02 03 46 ac cc 11 00 00 5e 00 01 00 08 00

root@nodek11:~# rt --dump 0 | grep 10.10.10.100 <=== IP-Fabric VN is configured as provider network, no ECMP here
10.10.10.100/32 32 PT - 21 -
root@nodek11:~# nh --get 21
Id:21 Type:Encap Fmly: AF_INET Rid:0 Ref_cnt:6 Vrf:0
              Flags:Valid, Policy, Etree Root,
              EncapFmly:0806 Oif:4 Len:14
              Encap Data: 02 03 46 ac cc 11 00 00 5e 00 01 00 08 00

root@nodek11:~#

[Nischal Sheth (nsheth)]

@haripk @ashoksr

Does this sound fixable or should we disallow active-active AAP
on VMIs in VNs that use ip-fabric as provider?

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/39932
Submitter: Naveen N (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/39932
Committed: http://github.com/Juniper/contrail-controller/commit/c0f91f524c1346afd17535c4d60b77a157c60b68
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit c0f91f524c1346afd17535c4d60b77a157c60b68
Author: Naveen N <email address hidden>
Date: Fri Feb 16 15:10:39 2018 +0530

* Initial changes to support ECMP in fabric VRF

1> Add routes in fabric VRF with VM peer, alllowing existing
ECMP path selection to happen
2> Add a flag in controller path to copy local path, as MPLS
label wouldnt be able to give nexthop
3> Upon a local path addition resync route such that control-node
path also gets updated
4> In case of fabric for egress flow ensure only local interface
gets picked up as part of ECMP hashing
Test for same.

Closes-bug: #1711508
Change-Id: I8acd0e29202a6216ab260bb4be28bf42ce4fb53f
(cherry picked from commit 0e73c795709869fb245587b7c602eee0b136c6f9)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/42925
Submitter: Ashok Singh (<email address hidden>)

[Jeba Paulaiyan (jebap)]

https://review.opencontrail.org/42925 is abandoned. hence moving to New