lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
$ ping 30.238.247.5
PING 30.238.247.5 (30.238.247.5): 56 data bytes
2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled)
$ ping 30.238.247.5
PING 30.238.247.5 (30.238.247.5): 56 data bytes
64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms
64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms
64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms
64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms
64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms
^C
--- 30.238.247.5 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.242/0.527/1.622 ms
$
3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled)
TEST : test_gw_less_fwd_single_vn_ip_fab
Test Gateway less forwarding with single VN and IP Fabric provider network. IP Fbaric network is configured as provider network
over vn1. Multiple VMs will be launched across compute nodes
and ping between VMs should be successful and happen through underlay. Ping between vhosts to VMs should be successful and happen
through underlay. Also, ping between VMs to vhosts also should be successful and should go through underlay.
2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper
n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-TestGWLessFWD-86069364'](fa89b6cb-4530-499c-83b4-559b6d9cb7b6)
2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLessFWD-86069364
2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLessFWD-86069364
> /contrail-test/common/gw_less_fwd/base.py(333)setup_gw_less_fwd()
-> vn_fixtures = self.setup_vns(vn)
2018-10-11 10:08:28,366 - INFO - Created VN vn1
2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1
11 10:08:33,292 - INFO - VMI 6b67ae72-9823-445c-ab4a-2f6f1c469ad9 verification in API Server passed
2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-61553487>]) created on node: (nodei8), Zone: (nova:nodei8)
2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-3bf2-4d62-9b53-732c41f0a5e2 verification in API Server passed
2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-33711132>]) created on node: (nodel9), Zone: (nova:nodel9)
2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-33711132 to be up..
2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLessFWD-86069364-33711132 passed
2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-61553487 to be up..
2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLessFWD-86069364-61553487 passed
--Return--
> /contrail-test/common/gw_less_fwd/base.py(351)setup_gw_less_fwd()->{'vm_fixtures': {'vm1': <VMFixtu...61553487>, 'vm2': <VMFixtu...33711132>}, 'vmi_fixtures': {'vmi1': <port_fi...f438ae10>, 'vmi2': <port_fi...f43a46d0>}, 'vn_fixtures': {'vn1': <vn_test...f438a6d0>}}
-> return ret_dict
(Pdb) n
> /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(63)test_gw_less_fwd_single_vn_ip_fab()
-> vn_fixtures = ret_dict['vn_fixtures']
(Pdb) n
> /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(70)test_gw_less_fwd_single_vn_ip_fab()
-> self.logger.info("-- SCENARIO: 1 Verifying gateway less forward "\
(Pdb) n
2018-10-11 10:14:21,372 - INFO - -- SCENARIO: 1 Verifying gateway less forward functionality without explicit policy --
> /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(72)test_gw_less_fwd_single_vn_ip_fab()
-> self.verify_gw_less_fwd(ret_dict)
(Pdb) n
2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169
2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164
2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4
2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4
[cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5
[cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes
[cirros@169.254.0.3] out:
[cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics ---
[cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss
[cirros@169.254.0.3] out:
Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'!
2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLessFWD-86069364-33711132 failed
[cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5
[cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes
[cirros@169.254.0.3] out:
[cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics ---
[cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss
[cirros@169.254.0.3] out:
Seeing the issue on Netronome testbed with acceleration enabled. Without acceleration, the issue is not observed.
Steps to recreate the issue:
1) Create a VN vn1
2) enable ip forwarding in vn1
Subnet(s)
CIDR Gateway Service Address DNS DHCP Allocation Pools
30.238.247.0/24 30.238.247.1 30.238.247.2 Enabled Enabled -
IP Fabric Forwarding Enabled
2) Launch 2 VMs in VN1
Ping from vm1 to vm2 and expect ping to work.
Observations:
1) Ping not working when gwless (ip fabric forwarding ) and netronome acceleration is enabled
^C 238.247. 255 Mask:255.255.255.0 f9ff:fea4: c93b/64 Scope:Link
collisions: 0 txqueuelen:1000
--- 30.238.247.5 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
$ ifconfig
eth0 Link encap:Ethernet HWaddr 02:04:F9:A4:C9:3B
inet addr:30.238.247.6 Bcast:30.
inet6 addr: fe80::4:
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:924 errors:0 dropped:0 overruns:0 frame:0
TX packets:962 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:87628 (85.5 KiB) TX bytes:92650 (90.4 KiB)
lo Link encap:Local Loopback
collisions: 0 txqueuelen:0
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
$ ping 30.238.247.5
PING 30.238.247.5 (30.238.247.5): 56 data bytes
2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled)
$ ping 30.238.247.5
PING 30.238.247.5 (30.238.247.5): 56 data bytes
64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms
64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms
64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms
64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms
64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms
^C
--- 30.238.247.5 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.242/0.527/1.622 ms
$
3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled)
TEST : test_gw_ less_fwd_ single_ vn_ip_fab
Test Gateway less forwarding with single VN and IP Fabric provider
network. IP Fbaric network is configured as provider network
underlay. Ping between vhosts to VMs should be successful and happen
successful and should go through underlay.
over vn1. Multiple VMs will be launched across compute nodes
and ping between VMs should be successful and happen through
through underlay. Also, ping between VMs to vhosts also should be
2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper TestGWLessFWD- 86069364' ](fa89b6cb- 4530-499c- 83b4-559b6d9cb7 b6) sFWD-86069364 sFWD-86069364 test/common/ gw_less_ fwd/base. py(333) setup_gw_ less_fwd( ) 9823-445c- ab4a-2f6f1c469a d9 verification in API Server passed sFWD-86069364- 61553487> ]) created on node: (nodei8), Zone: (nova:nodei8) 3bf2-4d62- 9b53-732c41f0a5 e2 verification in API Server passed sFWD-86069364- 33711132> ]) created on node: (nodel9), Zone: (nova:nodel9) sFWD-86069364- 33711132 to be up.. sFWD-86069364- 33711132 passed sFWD-86069364- 61553487 to be up.. sFWD-86069364- 61553487 passed test/common/ gw_less_ fwd/base. py(351) setup_gw_ less_fwd( )->{'vm_ fixtures' : {'vm1': <VMFixtu. ..61553487> , 'vm2': <VMFixtu. ..33711132> }, 'vmi_fixtures': {'vmi1': <port_fi. ..f438ae10> , 'vmi2': <port_fi. ..f43a46d0> }, 'vn_fixtures': {'vn1': <vn_test. ..f438a6d0> }} test/serial_ scripts/ gw_less_ fwd/test_ gw_less_ fwd.py( 63)test_ gw_less_ fwd_single_ vn_ip_fab( ) 'vn_fixtures' ] test/serial_ scripts/ gw_less_ fwd/test_ gw_less_ fwd.py( 70)test_ gw_less_ fwd_single_ vn_ip_fab( ) info("- - SCENARIO: 1 Verifying gateway less forward "\ test/serial_ scripts/ gw_less_ fwd/test_ gw_less_ fwd.py( 72)test_ gw_less_ fwd_single_ vn_ip_fab( ) gw_less_ fwd(ret_ dict) 169.254. 0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 169.254. 0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes 169.254. 0.3] out: 169.254. 0.3] out: --- 30.238.247.5 ping statistics --- 169.254. 0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss 169.254. 0.3] out:
n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-
2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLes
2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLes
> /contrail-
-> vn_fixtures = self.setup_vns(vn)
2018-10-11 10:08:28,366 - INFO - Created VN vn1
2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1
11 10:08:33,292 - INFO - VMI 6b67ae72-
2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLes
2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-
2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLes
2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLes
2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLes
2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLes
2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLes
--Return--
> /contrail-
-> return ret_dict
(Pdb) n
> /contrail-
-> vn_fixtures = ret_dict[
(Pdb) n
> /contrail-
-> self.logger.
(Pdb) n
2018-10-11 10:14:21,372 - INFO - -- SCENARIO: 1 Verifying gateway less forward functionality without explicit policy --
> /contrail-
-> self.verify_
(Pdb) n
2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169
2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164
2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4
2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4
[cirros@
[cirros@
[cirros@
[cirros@
[cirros@
[cirros@
Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'!
2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLes sFWD-86069364- 33711132 failed 169.254. 0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 169.254. 0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes 169.254. 0.3] out: 169.254. 0.3] out: --- 30.238.247.5 ping statistics --- 169.254. 0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss 169.254. 0.3] out:
[cirros@
[cirros@
[cirros@
[cirros@
[cirros@
[cirros@