Activity log for bug #1797358

Date Who What changed Old value New value Message
2018-10-11 10:59:39 Ankit Jain bug added bug
2018-10-11 10:59:50 Ankit Jain nominated for series juniperopenstack/r4.1
2018-10-11 10:59:50 Ankit Jain bug task added juniperopenstack/r4.1
2018-10-11 10:59:56 Ankit Jain juniperopenstack/r4.1: importance Undecided Critical
2018-10-11 11:00:06 Ankit Jain juniperopenstack/r4.1: assignee Pieter Malan (pmalan)
2018-10-11 11:01:09 Ankit Jain description Seeing the issue on Netronome testbed with acceleration enabled. Without acceleration, the issue is not observed. Steps to recreate the issue: 1) Create a VN vn1 2) enable ip forwarding in vn1 Subnet(s) CIDR Gateway Service Address DNS DHCP Allocation Pools 30.238.247.0/24 30.238.247.1 30.238.247.2 Enabled Enabled - IP Fabric Forwarding Enabled 2) Launch 2 VMs in VN1 Ping from vm1 to vm2 and expect ping to work. Observations: 1) Ping not working when gwless (ip fabric forwarding ) and netronome acceleration is enabled ^C --- 30.238.247.5 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ ifconfig eth0 Link encap:Ethernet HWaddr 02:04:F9:A4:C9:3B inet addr:30.238.247.6 Bcast:30.238.247.255 Mask:255.255.255.0 inet6 addr: fe80::4:f9ff:fea4:c93b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:924 errors:0 dropped:0 overruns:0 frame:0 TX packets:962 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:87628 (85.5 KiB) TX bytes:92650 (90.4 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms 64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms 64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms 64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms 64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms ^C --- 30.238.247.5 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 0.242/0.527/1.622 ms $ 3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled) TEST : test_gw_less_fwd_single_vn_ip_fab Test Gateway less forwarding with single VN and IP Fabric provider network. IP Fbaric network is configured as provider network over vn1. Multiple VMs will be launched across compute nodes and ping between VMs should be successful and happen through underlay. Ping between vhosts to VMs should be successful and happen through underlay. Also, ping between VMs to vhosts also should be successful and should go through underlay. 2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-TestGWLessFWD-86069364'](fa89b6cb-4530-499c-83b4-559b6d9cb7b6) 2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLessFWD-86069364 2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLessFWD-86069364 > /contrail-test/common/gw_less_fwd/base.py(333)setup_gw_less_fwd() -> vn_fixtures = self.setup_vns(vn) 2018-10-11 10:08:28,366 - INFO - Created VN vn1 2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1 11 10:08:33,292 - INFO - VMI 6b67ae72-9823-445c-ab4a-2f6f1c469ad9 verification in API Server passed 2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-61553487>]) created on node: (nodei8), Zone: (nova:nodei8) 2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-3bf2-4d62-9b53-732c41f0a5e2 verification in API Server passed 2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-33711132>]) created on node: (nodel9), Zone: (nova:nodel9) 2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-33711132 to be up.. 2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLessFWD-86069364-33711132 passed 2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-61553487 to be up.. 2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLessFWD-86069364-61553487 passed --Return-- > /contrail-test/common/gw_less_fwd/base.py(351)setup_gw_less_fwd()->{'vm_fixtures': {'vm1': <VMFixtu...61553487>, 'vm2': <VMFixtu...33711132>}, 'vmi_fixtures': {'vmi1': <port_fi...f438ae10>, 'vmi2': <port_fi...f43a46d0>}, 'vn_fixtures': {'vn1': <vn_test...f438a6d0>}} -> return ret_dict (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(63)test_gw_less_fwd_single_vn_ip_fab() -> vn_fixtures = ret_dict['vn_fixtures'] (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(70)test_gw_less_fwd_single_vn_ip_fab() -> self.logger.info("-- SCENARIO: 1 Verifying gateway less forward "\ (Pdb) n 2018-10-11 10:14:21,372 - INFO - -- SCENARIO: 1 Verifying gateway less forward functionality without explicit policy -- > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(72)test_gw_less_fwd_single_vn_ip_fab() -> self.verify_gw_less_fwd(ret_dict) (Pdb) n 2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169 2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164 2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'! 2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLessFWD-86069364-33711132 failed [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Seeing the issue on Netronome testbed with acceleration enabled. Without acceleration, the issue is not observed. Steps to recreate the issue: 1) Create a VN vn1 2) enable ip fabric forwarding in vn1 Subnet(s) CIDR Gateway Service Address DNS DHCP Allocation Pools 30.238.247.0/24 30.238.247.1 30.238.247.2 Enabled Enabled - IP Fabric Forwarding Enabled 2) Launch 2 VMs in VN1 Ping from vm1 to vm2 and expect ping to work. Observations: 1) Ping not working when gwless (ip fabric forwarding ) and netronome acceleration is enabled ^C --- 30.238.247.5 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ ifconfig eth0 Link encap:Ethernet HWaddr 02:04:F9:A4:C9:3B           inet addr:30.238.247.6 Bcast:30.238.247.255 Mask:255.255.255.0           inet6 addr: fe80::4:f9ff:fea4:c93b/64 Scope:Link           UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1           RX packets:924 errors:0 dropped:0 overruns:0 frame:0           TX packets:962 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:1000           RX bytes:87628 (85.5 KiB) TX bytes:92650 (90.4 KiB) lo Link encap:Local Loopback           inet addr:127.0.0.1 Mask:255.0.0.0           inet6 addr: ::1/128 Scope:Host           UP LOOPBACK RUNNING MTU:16436 Metric:1           RX packets:0 errors:0 dropped:0 overruns:0 frame:0           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:0           RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms 64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms 64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms 64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms 64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms ^C --- 30.238.247.5 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 0.242/0.527/1.622 ms $ 3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled)   TEST : test_gw_less_fwd_single_vn_ip_fab             Test Gateway less forwarding with single VN and IP Fabric provider             network. IP Fbaric network is configured as provider network             over vn1. Multiple VMs will be launched across compute nodes             and ping between VMs should be successful and happen through             underlay. Ping between vhosts to VMs should be successful and happen             through underlay. Also, ping between VMs to vhosts also should be             successful and should go through underlay. 2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-TestGWLessFWD-86069364'](fa89b6cb-4530-499c-83b4-559b6d9cb7b6) 2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLessFWD-86069364 2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLessFWD-86069364 > /contrail-test/common/gw_less_fwd/base.py(333)setup_gw_less_fwd() -> vn_fixtures = self.setup_vns(vn) 2018-10-11 10:08:28,366 - INFO - Created VN vn1 2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1 11 10:08:33,292 - INFO - VMI 6b67ae72-9823-445c-ab4a-2f6f1c469ad9 verification in API Server passed 2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-61553487>]) created on node: (nodei8), Zone: (nova:nodei8) 2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-3bf2-4d62-9b53-732c41f0a5e2 verification in API Server passed 2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-33711132>]) created on node: (nodel9), Zone: (nova:nodel9) 2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-33711132 to be up.. 2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLessFWD-86069364-33711132 passed 2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-61553487 to be up.. 2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLessFWD-86069364-61553487 passed --Return-- > /contrail-test/common/gw_less_fwd/base.py(351)setup_gw_less_fwd()->{'vm_fixtures': {'vm1': <VMFixtu...61553487>, 'vm2': <VMFixtu...33711132>}, 'vmi_fixtures': {'vmi1': <port_fi...f438ae10>, 'vmi2': <port_fi...f43a46d0>}, 'vn_fixtures': {'vn1': <vn_test...f438a6d0>}} -> return ret_dict (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(63)test_gw_less_fwd_single_vn_ip_fab() -> vn_fixtures = ret_dict['vn_fixtures'] (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(70)test_gw_less_fwd_single_vn_ip_fab() -> self.logger.info("-- SCENARIO: 1 Verifying gateway less forward "\ (Pdb) n 2018-10-11 10:14:21,372 - INFO - -- SCENARIO: 1 Verifying gateway less forward functionality without explicit policy -- > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(72)test_gw_less_fwd_single_vn_ip_fab() -> self.verify_gw_less_fwd(ret_dict) (Pdb) n 2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169 2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164 2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'! 2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLessFWD-86069364-33711132 failed [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out:
2018-10-11 11:04:39 Ankit Jain description Seeing the issue on Netronome testbed with acceleration enabled. Without acceleration, the issue is not observed. Steps to recreate the issue: 1) Create a VN vn1 2) enable ip fabric forwarding in vn1 Subnet(s) CIDR Gateway Service Address DNS DHCP Allocation Pools 30.238.247.0/24 30.238.247.1 30.238.247.2 Enabled Enabled - IP Fabric Forwarding Enabled 2) Launch 2 VMs in VN1 Ping from vm1 to vm2 and expect ping to work. Observations: 1) Ping not working when gwless (ip fabric forwarding ) and netronome acceleration is enabled ^C --- 30.238.247.5 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ ifconfig eth0 Link encap:Ethernet HWaddr 02:04:F9:A4:C9:3B           inet addr:30.238.247.6 Bcast:30.238.247.255 Mask:255.255.255.0           inet6 addr: fe80::4:f9ff:fea4:c93b/64 Scope:Link           UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1           RX packets:924 errors:0 dropped:0 overruns:0 frame:0           TX packets:962 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:1000           RX bytes:87628 (85.5 KiB) TX bytes:92650 (90.4 KiB) lo Link encap:Local Loopback           inet addr:127.0.0.1 Mask:255.0.0.0           inet6 addr: ::1/128 Scope:Host           UP LOOPBACK RUNNING MTU:16436 Metric:1           RX packets:0 errors:0 dropped:0 overruns:0 frame:0           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:0           RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms 64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms 64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms 64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms 64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms ^C --- 30.238.247.5 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 0.242/0.527/1.622 ms $ 3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled)   TEST : test_gw_less_fwd_single_vn_ip_fab             Test Gateway less forwarding with single VN and IP Fabric provider             network. IP Fbaric network is configured as provider network             over vn1. Multiple VMs will be launched across compute nodes             and ping between VMs should be successful and happen through             underlay. Ping between vhosts to VMs should be successful and happen             through underlay. Also, ping between VMs to vhosts also should be             successful and should go through underlay. 2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-TestGWLessFWD-86069364'](fa89b6cb-4530-499c-83b4-559b6d9cb7b6) 2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLessFWD-86069364 2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLessFWD-86069364 > /contrail-test/common/gw_less_fwd/base.py(333)setup_gw_less_fwd() -> vn_fixtures = self.setup_vns(vn) 2018-10-11 10:08:28,366 - INFO - Created VN vn1 2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1 11 10:08:33,292 - INFO - VMI 6b67ae72-9823-445c-ab4a-2f6f1c469ad9 verification in API Server passed 2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-61553487>]) created on node: (nodei8), Zone: (nova:nodei8) 2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-3bf2-4d62-9b53-732c41f0a5e2 verification in API Server passed 2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-33711132>]) created on node: (nodel9), Zone: (nova:nodel9) 2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-33711132 to be up.. 2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLessFWD-86069364-33711132 passed 2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-61553487 to be up.. 2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLessFWD-86069364-61553487 passed --Return-- > /contrail-test/common/gw_less_fwd/base.py(351)setup_gw_less_fwd()->{'vm_fixtures': {'vm1': <VMFixtu...61553487>, 'vm2': <VMFixtu...33711132>}, 'vmi_fixtures': {'vmi1': <port_fi...f438ae10>, 'vmi2': <port_fi...f43a46d0>}, 'vn_fixtures': {'vn1': <vn_test...f438a6d0>}} -> return ret_dict (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(63)test_gw_less_fwd_single_vn_ip_fab() -> vn_fixtures = ret_dict['vn_fixtures'] (Pdb) n > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(70)test_gw_less_fwd_single_vn_ip_fab() -> self.logger.info("-- SCENARIO: 1 Verifying gateway less forward "\ (Pdb) n 2018-10-11 10:14:21,372 - INFO - -- SCENARIO: 1 Verifying gateway less forward functionality without explicit policy -- > /contrail-test/serial_scripts/gw_less_fwd/test_gw_less_fwd.py(72)test_gw_less_fwd_single_vn_ip_fab() -> self.verify_gw_less_fwd(ret_dict) (Pdb) n 2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169 2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164 2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'! 2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLessFWD-86069364-33711132 failed [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Seeing the issue on Netronome testbed with acceleration enabled. Without acceleration, the issue is not observed. Steps to recreate the issue: 1) Create a VN vn1 2) enable ip fabric forwarding in vn1 Subnet(s) CIDR Gateway Service Address DNS DHCP Allocation Pools 30.238.247.0/24 30.238.247.1 30.238.247.2 Enabled Enabled - IP Fabric Forwarding Enabled 2) Launch 2 VMs in VN1 Ping from vm1 to vm2 and expect ping to work. Observations: 1) Ping not working when gwless (ip fabric forwarding ) and netronome acceleration is enabled ^C --- 30.238.247.5 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ ifconfig eth0 Link encap:Ethernet HWaddr 02:04:F9:A4:C9:3B           inet addr:30.238.247.6 Bcast:30.238.247.255 Mask:255.255.255.0           inet6 addr: fe80::4:f9ff:fea4:c93b/64 Scope:Link           UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1           RX packets:924 errors:0 dropped:0 overruns:0 frame:0           TX packets:962 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:1000           RX bytes:87628 (85.5 KiB) TX bytes:92650 (90.4 KiB) lo Link encap:Local Loopback           inet addr:127.0.0.1 Mask:255.0.0.0           inet6 addr: ::1/128 Scope:Host           UP LOOPBACK RUNNING MTU:16436 Metric:1           RX packets:0 errors:0 dropped:0 overruns:0 frame:0           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:0           RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 2) Ping starts working on disabling ip fabric forwarding ( Netronome acceleration is still enabled) $ ping 30.238.247.5 PING 30.238.247.5 (30.238.247.5): 56 data bytes 64 bytes from 30.238.247.5: seq=0 ttl=63 time=1.622 ms 64 bytes from 30.238.247.5: seq=1 ttl=63 time=0.248 ms 64 bytes from 30.238.247.5: seq=2 ttl=63 time=0.283 ms 64 bytes from 30.238.247.5: seq=3 ttl=63 time=0.242 ms 64 bytes from 30.238.247.5: seq=4 ttl=63 time=0.243 ms ^C --- 30.238.247.5 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 0.242/0.527/1.622 ms $ 3) Ping works on disabling Netronome acceleration ( ip fabric forwarding is still enabled)   TEST : test_gw_less_fwd_single_vn_ip_fab             Test Gateway less forwarding with single VN and IP Fabric provider             network. IP Fbaric network is configured as provider network             over vn1. Multiple VMs will be launched across compute nodes             and ping between VMs should be successful and happen through             underlay. Ping between vhosts to VMs should be successful and happen             through underlay. Also, ping between VMs to vhosts also should be             successful and should go through underlay. 2018-10-11 10:08:25,562 - INFO - Created BGP router: hooper n2018-10-11 10:08:25,959 - INFO - Using existing project [u'default-domain', u'ctest-TestGWLessFWD-86069364'](fa89b6cb-4530-499c-83b4-559b6d9cb7b6) 2018-10-11 10:08:25,962 - INFO - Default SG to be edited for allow all on project: ctest-TestGWLessFWD-86069364 2018-10-11 10:08:26,037 - INFO - Adding rules to the default security group in Project ctest-TestGWLessFWD-86069364 > /contrail-test/common/gw_less_fwd/base.py(333)setup_gw_less_fwd() -> vn_fixtures = self.setup_vns(vn) 2018-10-11 10:08:28,366 - INFO - Created VN vn1 2018-10-11 10:08:28,656 - INFO - Configuring IP Fabric forwarding on VN: vn1 11 10:08:33,292 - INFO - VMI 6b67ae72-9823-445c-ab4a-2f6f1c469ad9 verification in API Server passed 2018-10-11 10:08:35,759 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-61553487>]) created on node: (nodei8), Zone: (nova:nodei8) 2018-10-11 10:08:37,729 - INFO - VMI 04f9a4c9-3bf2-4d62-9b53-732c41f0a5e2 verification in API Server passed 2018-10-11 10:08:41,565 - INFO - VM ([<Server: ctest-TestGWLessFWD-86069364-33711132>]) created on node: (nodel9), Zone: (nova:nodel9) 2018-10-11 10:08:41,567 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-33711132 to be up.. 2018-10-11 10:08:55,864 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestGWLessFWD-86069364-33711132 passed 2018-10-11 10:09:01,067 - INFO - Waiting for VM ctest-TestGWLessFWD-86069364-61553487 to be up.. 2018-10-11 10:09:03,992 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestGWLessFWD-86069364-61553487 passed --Return-- > /contrail-test/common/gw_less_fwd/base.py(351)setup_gw_less_fwd()->{'vm_fixtures': {'vm1': <VMFixtu...61553487>, 'vm2': <VMFixtu...33711132>}, 'vmi_fixtures': {'vmi1': <port_fi...f438ae10>, 'vmi2': <port_fi...f43a46d0>}, 'vn_fixtures': {'vn1': <vn_test...f438a6d0>}} 2018-10-11 10:14:23,074 - INFO - Route: 30.238.247.6 is found in default routing instance as expected in agent on compute node: 192.168.11.169 2018-10-11 10:14:23,165 - INFO - Route: 30.238.247.5 is found in default routing instance as expected in agent on compute node: 192.168.11.164 2018-10-11 10:14:23,474 - INFO - Route: 30.238.247.6 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 2018-10-11 10:14:23,724 - INFO - Route: 30.238.247.5 is found in defaultrouting instance as expected incontrol node: 192.168.31.4 [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out: Warning: sudo() received nonzero return code 1 while executing 'ping -s 56 -c 2 -W 1 30.238.247.5'! 2018-10-11 10:14:28,020 - WARNING - Ping to IP 30.238.247.5 from VM ctest-TestGWLessFWD-86069364-33711132 failed [cirros@169.254.0.3] sudo: ping -s 56 -c 2 -W 1 30.238.247.5 [cirros@169.254.0.3] out: PING 30.238.247.5 (30.238.247.5): 56 data bytes [cirros@169.254.0.3] out: [cirros@169.254.0.3] out: --- 30.238.247.5 ping statistics --- [cirros@169.254.0.3] out: 2 packets transmitted, 0 packets received, 100% packet loss [cirros@169.254.0.3] out:
2018-10-11 11:23:24 Pieter Malan juniperopenstack/r4.1: assignee Pieter Malan (pmalan)
2018-10-11 11:24:12 Pieter Malan juniperopenstack/r4.1: assignee Pieter Malan (pmalan)
2018-10-11 11:24:55 Pieter Malan bug added subscriber Frikkie Scholtz
2018-10-11 14:08:35 Pieter Malan bug added subscriber Frik Botha
2018-10-11 18:45:54 Sudheendra Rao juniperopenstack/r4.1: milestone r4.1.2.0
2018-10-11 18:46:03 Sudheendra Rao juniperopenstack: importance Undecided Critical
2018-10-11 18:46:11 Sudheendra Rao nominated for series juniperopenstack/trunk
2018-10-11 18:46:11 Sudheendra Rao bug task added juniperopenstack/trunk
2018-10-11 18:47:29 Sudheendra Rao juniperopenstack/trunk: assignee Pieter Malan (pmalan)
2018-10-11 18:48:18 Sudheendra Rao tags netronome vrouter blocker netronome
2018-10-11 20:32:32 Jeba Paulaiyan juniperopenstack/trunk: milestone r5.1.0
2018-10-22 21:37:57 Jeba Paulaiyan juniperopenstack/r4.1: importance Critical High
2018-10-22 21:37:59 Jeba Paulaiyan juniperopenstack/trunk: importance Critical High
2018-10-22 21:38:04 Jeba Paulaiyan juniperopenstack/r4.1: milestone r4.1.2.0
2018-10-24 16:41:06 Sudheendra Rao tags blocker netronome blocker netronome releasenote
2018-10-24 16:41:22 Sudheendra Rao juniperopenstack/r4.1: milestone r4.1.2.0
2018-10-24 16:41:32 Sudheendra Rao juniperopenstack/r4.1: importance High Medium
2018-10-24 16:41:34 Sudheendra Rao juniperopenstack/trunk: importance High Medium
2018-10-25 16:18:06 Jeba Paulaiyan juniperopenstack/r4.1: milestone r4.1.2.0
2018-10-27 02:47:51 Sudheendra Rao juniperopenstack/r4.1: milestone r4.1.2.0
2018-10-27 02:47:56 Sudheendra Rao juniperopenstack/r4.1: status New Won't Fix
2018-10-27 02:47:59 Sudheendra Rao juniperopenstack/trunk: status New Won't Fix