l3 ping test failure

[XuRong]

1.Configuring two private networks through the openstack dashboard
2.launch an instance in each private network
3.add the two interfaces to router
vm1 ping v2 failure

example:

we use network20 network30 and router_test

[root@devstack-152 networking-ovn]# neutron net-list
+--------------------------------------+-------------+----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+-------------+----------------------------------------------------+
| f79d123a-cd13-428d-a962-7cd58a28fc3d | providernet | 098ae10b-79a4-4303-a3ae-2c23b5a135d6 9.0.0.0/24 |
| c1c99fff-7673-4081-ac69-32871d2db01a | public | 217d79f1-f7e3-4a74-b1e4-e3d41fd27b4f 172.24.4.0/24 |
| 8fc6c5b2-367f-468a-9d68-c4e5df594882 | network30 | 4ca5f8cc-68a7-4850-96c2-256312be8c75 30.0.0.0/24 |
| 19e91d9b-30bd-4478-9b9b-689f0d01f6ce | network20 | 5aa944f4-dafb-4328-aa99-93b7241dd192 20.0.0.0/24 |
+--------------------------------------+-------------+----------------------------------------------------+
[root@devstack-152 networking-ovn]# neutron subnet-list
+------------------------------------+-----------------+---------------+-------------------------------------+
| id | name | cidr | allocation_pools |
+------------------------------------+-----------------+---------------+-------------------------------------+
| 098ae10b-79a4-4303-a3ae- | provider-subnet | 9.0.0.0/24 | {"start": "9.0.0.2", "end": |
| 2c23b5a135d6 | | | "9.0.0.254"} |
| 217d79f1-f7e3-4a74-b1e4-e3d41fd27b | public-subnet | 172.24.4.0/24 | {"start": "172.24.4.2", "end": |
| 4f | | | "172.24.4.254"} |
| 4ca5f8cc- | subnet30 | 30.0.0.0/24 | {"start": "30.0.0.2", "end": |
| 68a7-4850-96c2-256312be8c75 | | | "30.0.0.254"} |
| 5aa944f4-dafb-4328-aa99-93b7241dd1 | subnet20 | 20.0.0.0/24 | {"start": "20.0.0.2", "end": |
| 92 | | | "20.0.0.254"} |
+------------------------------------+-----------------+---------------+-------------------------------------+
[root@devstack-152 networking-ovn]# neutron router-list
+--------------------------------------+-------------+-----------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+-------------+-----------------------------------------------+
| 9cf5e057-5928-40b4-85e2-b1fd3e229ccb | router1 | {"network_id": |
| | | "c1c99fff-7673-4081-ac69-32871d2db01a", |
| | | "external_fixed_ips": [{"subnet_id": |
| | | "217d79f1-f7e3-4a74-b1e4-e3d41fd27b4f", |
| | | "ip_address": "172.24.4.11"}]} |
| e7224fea-c58c-46ec-b024-947dddee464d | router_test | {"network_id": |
| | | "c1c99fff-7673-4081-ac69-32871d2db01a", |
| | | "external_fixed_ips": [{"subnet_id": |
| | | "217d79f1-f7e3-4a74-b1e4-e3d41fd27b4f", |
| | | "ip_address": "172.24.4.12"}]} |
+--------------------------------------+-------------+-----------------------------------------------+
[root@devstack-152 networking-ovn]# neutron router-port-list router_test
+--------------------------------------+------+-------------------+----------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------+
| 2215bba3-35d5-4fc4-aaa7-1c511d01ab5b | | fa:16:3e:01:62:63 | {"subnet_id": "5aa944f4-dafb-4328-aa99 |
| | | | -93b7241dd192", "ip_address": |
| | | | "20.0.0.1"} |
| c0e4f345-c207-47e6-a604-bd857238ff7a | | fa:16:3e:c3:7c:a8 | {"subnet_id": "098ae10b-79a4-4303 |
| | | | -a3ae-2c23b5a135d6", "ip_address": |
| | | | "9.0.0.1"} |
| d322b3fd-9330-4b9d-bc77-f29b142db1a3 | | fa:16:3e:92:0b:4e | {"subnet_id": "217d79f1-f7e3-4a74-b1e4 |
| | | | -e3d41fd27b4f", "ip_address": |
| | | | "172.24.4.12"} |
| e11f2c51-ba47-47f3-8842-192d42c815e0 | | fa:16:3e:ef:fe:6c | {"subnet_id": "4ca5f8cc- |
| | | | 68a7-4850-96c2-256312be8c75", |
| | | | "ip_address": "30.0.0.1"} |
+--------------------------------------+------+-------------------+----------------------------------------+
[root@devstack-152 devstack]# nova list
+--------------------------------------+--------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+--------+--------+------------+-------------+---------------------+
| 8eba0170-a7a5-498c-b60f-e878b98e2fba | vm20-1 | ACTIVE | - | Running | network20=20.0.0.10 |
| caef102c-88ed-4a92-8488-a0aa3d9efac0 | vm30-1 | ACTIVE | - | Running | network30=30.0.0.7 |
+--------------------------------------+--------+--------+------------+-------------+---------------------+

vm20-1 ping vm30-1 failed .

But the same condition is good on July 11,why?

[Numan Siddique (numansiddique)]

You are using native ovn l3 or l3 agent ?

[Richard Theis (rtheis)]

If you are using native L3, you may need https://review.openstack.org/#/c/344078/

[XuRong (xu-rong)]

Yes,I am using native ovn L3

[ZongKai LI (zongkai)]

Hi, Richard and Rong.
Yes, this bug is related to https://review.openstack.org/#/c/344078/.
Empty Logical_Router_Port.networks column will lead ovn-northd fail to parse and generate ovn_port entry for logical router port. Further, it will cause patch port for router and ovs flows for ip routing missed. So that's the reason why routing failed now.

Since patch to support set as column value type didn't get any response, shall we merge the revert patch for now? It makes feel bad to trouble people for an known issue. :(

[Richard Theis (rtheis)]

Hi ZongKai, if we can't get the OVS patch within a couple days. I think we should proceed with the revert.

[Russell Bryant (russellb)]

I will apply the ovs patch in a few minutes

[Richard Theis (rtheis)]

The ovs patch has been committed: https://github.com/openvswitch/ovs/commit/38c44378433b377bea6d4ff0fa4412a356cbfad3

[Richard Theis (rtheis)]

The ovs patch has been released https://pypi.python.org/pypi/ovs/2.6.0.dev2

[Richard Theis (rtheis)]

Hi ZongKai, we still need the revert (https://review.openstack.org/#/c/344078/) because ovs 2.5.0 is used in python 2.7 environment per the OpenStack requirements.

[Numan Siddique (numansiddique)]

Please note that the ovs python master and the 2.6 (https://pypi.python.org/pypi/ovs/2.6.0.dev2) is broken. As networking-ovn CI uses the ovs and ovn master code, the ovs python IDL will use the ovsdb update2 method by default. The recent patches to use update2 in ovs python has a bug because of which networking_ovn/ovsdb/ovsdb_monitor.py will not get the port up and down notification as expected.
Please see [1] for more details.

I have submitted a patch [2] to use the ovs python from master to confirm the failures. If we want to use the fix - "python: add set type for ovs.idl.data.Datum.from_python" we also need [1] to be merged.

[1] - https://patchwork.ozlabs.org/patch/651140/
[2] - https://review.openstack.org/#/c/280591/4

[Richard Theis (rtheis)]

Thank you Numan. I think we are goind with a fix for now: https://review.openstack.org/#/c/344078/

[Richard Theis (rtheis)]

This should now be fixed on master after https://review.openstack.org/#/c/344078/ merged.