Gateway_less_Fwd: Old flow continues with overlay even after configuring IP-Fabric as provider network
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Juniper Openstack | Status tracked in Trunk | |||||
R4.1 |
Fix Committed
|
Medium
|
Naveen N | |||
Trunk |
Fix Committed
|
Medium
|
Naveen N |
Bug Description
Old flow continues with overlay (instead of underlay) even after configuring IP-Fabric as provider network.
Steps
------
1. Configure a VN
2. Bring up 2 VMs across 2 compute nodes within above VN
3. Initiate a ping between Vms and verify the traffic is overlay
4. Now, configure IP-Fabric network as provider network on above VN. Now, traffic should be underlay
5. Now, seeing old flow still continues with overlay, which is incorrect
Please see the log below:
root@nodek11:~# contrail-version
Package Version Build-ID | Repo | Package Name
-------
contrail-lib 4.1.0.0-16 16
contrail-nodemgr 4.1.0.0-16 16
contrail-setup 4.1.0.0-16 16
contrail-utils 4.1.0.0-16 16
contrail-
contrail-
contrail-
contrail-
contrail-
nova-common 2:13.0.
nova-compute 2:13.0.
nova-compute-kvm 2:13.0.
nova-compute-
python-contrail 4.1.0.0-16 16
python-
python-
python-nova 2:13.0.
python-
root@nodek11:~# vif --list
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root
vif0/0 OS: em1 (Speed 1000, Duplex 1)
Vrf:0 Flags:L3L2VpEr QOS:-1 Ref:6
RX packets:5846 bytes:1113218 errors:0
TX packets:3534 bytes:2045434 errors:0
vif0/1 OS: vhost0
Vrf:0 Flags:PL3DEr QOS:-1 Ref:7
RX packets:3843 bytes:2051500 errors:0
TX packets:6312 bytes:1133014 errors:0
Drops:1
vif0/2 OS: pkt0
RX packets:908 bytes:130842 errors:0
TX packets:2842 bytes:312746 errors:0
Drops:0
vif0/3 OS: tapa767c4dd-5d
Vrf:2 Flags:PL3L2DEr QOS:-1 Ref:5
RX packets:661 bytes:63626 errors:0
TX packets:970 bytes:95278 errors:0
vif0/4350 OS: pkt3
RX packets:24 bytes:2352 errors:0
TX packets:24 bytes:2016 errors:0
Drops:0
vif0/4351 OS: pkt1
RX packets:0 bytes:0 errors:0
TX packets:0 bytes:0 errors:0
Drops:0
root@nodek11:~# rt --dump 2 | grep 10.10.10.4
10.10.10.4/32 32 LP 22 20 2:4f:22:
10.10.10.40/32 24 TF - 1 -
10.10.10.41/32 24 TF - 1 -
10.10.10.42/32 24 TF - 1 -
10.10.10.43/32 24 TF - 1 -
10.10.10.44/32 24 TF - 1 -
10.10.10.45/32 24 TF - 1 -
10.10.10.46/32 24 TF - 1 -
10.10.10.47/32 24 TF - 1 -
10.10.10.48/32 24 TF - 1 -
10.10.10.49/32 24 TF - 1 -
root@nodek11:~# nh --get 20
Id:20 Type:Tunnel Fmly: AF_INET Rid:0 Ref_cnt:7 Vrf:0
Oif:0 Len:14 Data:00 25 90 93 d2 44 0c c4 7a 32 0a 88 08 00
root@nodek11:~#
root@nodek11:~#
root@nodek11:~# flow --match 10.10.10.4
Flow table(size 80609280, entries 629760)
Entries: Created 322 Added 322 Deleted 430 Changed 434 Processed 322 Used Overflow entries 0
(Created Flows/CPU: 22 21 13 28 22 36 14 23 6 13 9 6 1 4 1 3 5 5 41 2 20 4 8 9 0 2 0 4 0 0 0 0)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Listing flows matching ([10.10.10.4]:*)
Index Source:
-------
290208<=>315276 10.10.10.3:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):32, Stats:48/4704,
SPort 61366, TTL 0, Sinfo 3.0.0.0)
315276<=>290208 10.10.10.4:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):20, Stats:48/4704,
SPort 55652, TTL 0, Sinfo 10.204.216.34)
root@nodek11:~# flow --match 10.10.10.4
Flow table(size 80609280, entries 629760)
Entries: Created 322 Added 322 Deleted 434 Changed 438 Processed 322 Used Overflow entries 0
(Created Flows/CPU: 22 21 13 28 22 36 14 23 6 13 9 6 1 4 1 3 5 5 41 2 20 4 8 9 0 2 0 4 0 0 0 0)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Listing flows matching ([10.10.10.4]:*)
Index Source:
-------
290208<=>315276 10.10.10.3:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):32, Stats:53/5194,
SPort 61366, TTL 0, Sinfo 3.0.0.0)
315276<=>290208 10.10.10.4:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):20, Stats:53/5194,
SPort 55652, TTL 0, Sinfo 10.204.216.34)
root@nodek11:~#
root@nodek11:~#
root@nodek11:~# tcpdump -ne -i em1 host 10.204.216.34 -vvv -xxx <==== Overlay traffic
tcpdump: WARNING: em1: no IPv4 address assigned
tcpdump: listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
11:59:40.178800 0c:c4:7a:32:0a:88 > 00:25:90:93:d2:44, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 130)
10.
0x0000: 0025 9093 d244 0cc4 7a32 0a88 0800 4500
0x0010: 0082 0000 0000 4011 b3c9 0acc d8e7 0acc
0x0020: d822 efb6 19eb 006e 0000 0001 9140 024f
0x0030: 22ad ab5d 02a7 67c4 dd5d 0800 4500 0054
0x0040: 0000 4000 4001 128f 0a0a 0a03 0a0a 0a04
0x0050: 0800 678d 076d 004b 53e6 9359 0000 0000
0x0060: d8a7 0a00 0000 0000 1011 1213 1415 1617
0x0070: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627
0x0080: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637
11:59:40.179245 00:25:90:93:d2:44 > 0c:c4:7a:32:0a:88, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 64, id 3282, offset 0, flags [none], proto UDP (17), length 130)
10.
0x0000: 0cc4 7a32 0a88 0025 9093 d244 0800 4500
0x0010: 0082 0cd2 0000 4011 a6f7 0acc d822 0acc
0x0020: d8e7 cc98 19eb 006e 0000 0001 f140 02a7
0x0030: 67c4 dd5d 024f 22ad ab5d 0800 4500 0054
0x0040: 0cd2 0000 4001 45bd 0a0a 0a04 0a0a 0a03
0x0050: 0000 6f8d 076d 004b 53e6 9359 0000 0000
0x0060: d8a7 0a00 0000 0000 1011 1213 1415 1617
0x0070: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627
0x0080: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637
11:59:40.815703 00:25:90:93:d2:44 > 0c:c4:7a:32:0a:88, ethertype IPv4 (0x0800), length 176: (tos 0x0, ttl 64, id 839, offset 0, flags [none], proto UDP (17), length 162)
10.
0x0000: 0cc4 7a32 0a88 0025 9093 d244 0800 4500
0x0010: 00a2 0347 0000 4011 b062 0acc d822 0acc
0x0020: d8e7 c240 19eb 008e 0000 0120 9140 0000
0x0030: 0000 4500 007e 0345 0000 4011 b14d 0acc
0x0040: d822 0acc d822 c240 12b5 006a 0000 0800
0x0050: 0000 0000 0400 3333 0000 0016 024f 22ad
0x0060: ab5d 86dd 6000 0000 0024 0001 fe80 0000
0x0070: 0000 0000 004f 22ff fead ab5d ff02 0000
0x0080: 0000 0000 0000 0000 0000 0016 3a00 0502
0x0090: 0000 0100 8f00 f7a4 0000 0001 0400 0000
0x00a0: ff02 0000 0000 0000 0000 0001 ffad ab5d
^C
3 packets captured
7 packets received by filter
0 packets dropped by kernel
root@nodek11:~#
root@nodek11:~#
root@nodek11:~#
root@nodek11:~#
root@nodek11:~# vif --list
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root
vif0/0 OS: em1 (Speed 1000, Duplex 1)
Vrf:0 Flags:L3L2VpEr QOS:-1 Ref:6
RX packets:8684 bytes:1465930 errors:0
TX packets:4855 bytes:2713499 errors:0
vif0/1 OS: vhost0
Vrf:0 Flags:PL3DEr QOS:-1 Ref:7
RX packets:5101 bytes:2691795 errors:0
TX packets:9111 bytes:1479104 errors:0
Drops:1
vif0/2 OS: pkt0
RX packets:996 bytes:142910 errors:0
TX packets:3945 bytes:430614 errors:0
Drops:0
vif0/3 OS: tapa767c4dd-5d
Vrf:2 Flags:PL3L2DEr QOS:-1 Ref:5
RX packets:1208 bytes:130592 errors:0
TX packets:1515 bytes:142348 errors:0
vif0/4350 OS: pkt3
RX packets:231 bytes:22638 errors:0
TX packets:231 bytes:19404 errors:0
Drops:0
vif0/4351 OS: pkt1
RX packets:0 bytes:0 errors:0
TX packets:0 bytes:0 errors:0
Drops:0
root@nodek11:~#
root@nodek11:~# vif --get 3
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root
vif0/3 OS: tapa767c4dd-5d
Vrf:2 Flags:PL3L2DEr QOS:-1 Ref:5 <=== vrf:2
RX packets:1228 bytes:133054 errors:0
TX packets:1537 bytes:144204 errors:0
root@nodek11:~#
-------
Now, configuring IP-Fabric as provider network for left_vn
-------
>>> from vnc_api import vnc_api;from pprint import pprint;vnc_lib = vnc_api.
>>>
>>>
>>>
>>>
>>>
>>>
>>> pprint(
{u'virtual-
>>>
>>>
>>>
>>> ip_fab_vn = vnc_lib.
>>> left_vn = vnc_lib.
>>>
>>> left_vn.
>>> vnc_lib.
u'{"virtual-
>>>
root@nodek11:~# vif --get 3 <==== Now, it belongs to IP-Fabric VN, which is correct
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root
vif0/3 OS: tapa767c4dd-5d
Vrf:0 Flags:PL3L2DProxyEr QOS:-1 Ref:5 <=== Vrf:0
RX packets:1304 bytes:141798 errors:0
TX packets:1611 bytes:150240 errors:0
root@nodek11:~# flow --match 10.10.10.4
Flow table(size 80609280, entries 629760)
Entries: Created 322 Added 322 Deleted 478 Changed 483 Processed 322 Used Overflow entries 0
(Created Flows/CPU: 22 21 13 28 22 36 14 23 6 13 9 6 1 4 1 3 5 5 41 2 20 4 8 9 0 2 0 4 0 0 0 0)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Listing flows matching ([10.10.10.4]:*)
Index Source:
-------
290208<=>315276 10.10.10.3:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):32, Stats:284/27832,
SPort 61366, TTL 0, Sinfo 3.0.0.0)
315276<=>290208 10.10.10.4:1901 1 (2)
(Gen: 1, K(nh):32, Action:F, Flags:, QOS:-1, S(nh):20, Stats:259/25382,
SPort 55652, TTL 0, Sinfo 10.204.216.34)
root@nodek11:~#
root@nodek11:~# tcpdump -ne -i em1 host 10.204.216.34 -vvv -xxx <==== Still, overlay (instaed of underlay)
tcpdump: WARNING: em1: no IPv4 address assigned
tcpdump: listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
12:03:25.174901 0c:c4:7a:32:0a:88 > 00:25:90:93:d2:44, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 130)
10.
0x0000: 0025 9093 d244 0cc4 7a32 0a88 0800 4500
0x0010: 0082 0000 0000 4011 b3c9 0acc d8e7 0acc
0x0020: d822 efb6 19eb 006e 0000 0001 9140 024f
0x0030: 22ad ab5d 02a7 67c4 dd5d 0800 4500 0054
0x0040: 0000 4000 4001 128f 0a0a 0a03 0a0a 0a04
0x0050: 0800 e09b 076d 012c 34e7 9359 0000 0000
0x0060: 7db7 0a00 0000 0000 1011 1213 1415 1617
0x0070: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627
0x0080: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637
12:03:26.174864 0c:c4:7a:32:0a:88 > 00:25:90:93:d2:44, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 130)
10.
0x0000: 0025 9093 d244 0cc4 7a32 0a88 0800 4500
0x0010: 0082 0000 0000 4011 b3c9 0acc d8e7 0acc
0x0020: d822 efb6 19eb 006e 0000 0001 9140 024f
0x0030: 22ad ab5d 02a7 67c4 dd5d 0800 4500 0054
0x0040: 0000 4000 4001 128f 0a0a 0a03 0a0a 0a04
0x0050: 0800 df9a 076d 012d 35e7 9359 0000 0000
0x0060: 7db7 0a00 0000 0000 1011 1213 1415 1617
0x0070: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627
0x0080: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637
12:03:27.174828 0c:c4:7a:32:0a:88 > 00:25:90:93:d2:44, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 130)
10.
0x0000: 0025 9093 d244 0cc4 7a32 0a88 0800 4500
0x0010: 0082 0000 0000 4011 b3c9 0acc d8e7 0acc
0x0020: d822 efb6 19eb 006e 0000 0001 9140 024f
0x0030: 22ad ab5d 02a7 67c4 dd5d 0800 4500 0054
0x0040: 0000 4000 4001 128f 0a0a 0a03 0a0a 0a04
0x0050: 0800 e199 076d 012e 36e7 9359 0000 0000
0x0060: 7ab7 0a00 0000 0000 1011 1213 1415 1617
0x0070: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627
0x0080: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637
^C
3 packets captured
5 packets received by filter
0 packets dropped by kernel
root@nodek11:~#
summary: |
- IP_Fabric_Fwd: Old flow continues with overlay even after configuring + Gateway_less_Fwd: Old flow continues with overlay even after configuring IP-Fabric as provider network |
information type: | Proprietary → Public |
Not planning to fix this for R4.1.