Functional test test_gateway_chassis_rebalance failing due to "failed to bind logical router"

Bug #1918266 reported by Slawek Kaplonski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
High
Jakub Libosvar

Bug Description

Error example: https://40b5766ce602bfb4b663-445d4465f34d2b24df5d805a76ff9803.ssl.cf1.rackcdn.com/765846/7/check/neutron-functional-with-uwsgi/928e7cf/testr_results.html

Stacktrace:
ft1.14: neutron.tests.functional.services.ovn_l3.test_plugin.TestRouter.test_gateway_chassis_rebalancetesttools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
    return f(self, *args, **kwargs)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/services/ovn_l3/test_plugin.py", line 497, in test_gateway_chassis_rebalance
    self.assertTrue(self.cr_lrp_pb_event.wait(logical_port),
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/unittest2/case.py", line 702, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true : lrp cr-lrp-488b3887-1770-4a72-86cb-7306e78c954a failed to bind

Logstash query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20497%2C%20in%20test_gateway_chassis_rebalance%5C%22

Changed in neutron:
assignee: nobody → Jakub Libosvar (libosvar)
Revision history for this message
Jakub Libosvar (libosvar) wrote :
Download full text (7.7 KiB)

I inspected the logs and it seems there are problems with the transaction order:

What happens is that during router creation, we create one transaction and add commands to create logical router and then to create logical router port for the gateway, in this order. However, in NB DB, the logical router port is created before:

2021-03-08T22:51:15.603Z|01041|jsonrpc|DBG|unix#5: received request, method="transact", params=["OVN_Northbound",{"uuid-name":"rowc664e9f4_477a_42b9_94e1_1cfd75e37232","table":"Logical_Switch_Port","op":"insert","row":{"name":"488b3887-1770-4a72-86cb-7306e78c954a","addresses":["set",["12:34:56:78:84:c3 30.0.0.35","unknown"]],"options":["map",[["requested-chassis",""]]],"external_ids":["map",[["neutron:cidrs","30.0.0.35/24"],["neutron:device_id","0eff9eae-4642-4abe-9c06-67df13921296"],["neutron:device_owner","network:router_gateway"],["neutron:network_name","neutron-a0ce1add-b6ec-41df-bf85-44bac81781ed"],["neutron:port_name",""],["neutron:project_id",""],["neutron:revision_number","1"],["neutron:security_group_ids",""]]],"type":"","enabled":true}},{"where":[["_uuid","==",["uuid","6317396c-75ed-457b-a75b-09812e134099"]]],"mutations":[["ports","insert",["set",[["named-uuid","rowc664e9f4_477a_42b9_94e1_1cfd75e37232"]]]]],"table":"Logical_Switch","op":"mutate"}], id=1958

2021-03-08T22:51:17.278Z|01069|jsonrpc|DBG|unix#5: received request, method="transact", params=["OVN_Northbound",{"uuid-name":"rowd215a94b_9275_4fd0_b73c_c2c7e864379a","table":"Logical_Router","op":"insert","row":{"name":"neutron-0eff9eae-4642-4abe-9c06-67df13921296","external_ids":["map",[["neutron:availability_zone_hints",""],["neutron:gw_port_id","488b3887-1770-4a72-86cb-7306e78c954a"],["neutron:revision_number","2"],["neutron:router_name","router14"]]],"enabled":true}},{"where":[["_uuid","==",["named-uuid","rowd215a94b_9275_4fd0_b73c_c2c7e864379a"]]],"mutations":[["ports","insert",["set",[["named-uuid","row34af8860_ab43_4091_ac45_d832a818fa95"]]]],["static_routes","insert",["set",[["named-uuid","row9d1df734_4604_4cd5_a431_855104e1c2b7"]]]]],"table":"Logical_Router","op":"mutate"},{"uuid-name":"row34af8860_ab43_4091_ac45_d832a818fa95","table":"Logical_Router_Port","op":"insert","row":{"name":"lrp-488b3887-1770-4a72-86cb-7306e78c954a","mac":"12:34:56:78:84:c3","external_ids":["map",[["neutron:network_name","neutron-a0ce1add-b6ec-41df-bf85-44bac81781ed"],["neutron:revision_number","1"],["neutron:router_name","0eff9eae-4642-4abe-9c06-67df13921296"],["neutron:subnet_ids","4f8bbafc-ef7c-41bf-917d-cfd52b105586"]]],"gateway_chassis":["named-uuid","row759e3421_f7a6_4d0b_9d43_ffd4ce28158d"],"networks":"30.0.0.35/24"}},{"uuid-name":"row759e3421_f7a6_4d0b_9d43_ffd4ce28158d","table":"Gateway_Chassis","op":"insert","row":{"name":"lrp-488b3887-1770-4a72-86cb-7306e78c954a_92ca516b-a624-42e1-8460-5dcf6b57b98c","priority":1,"chassis_name":"92ca516b-a624-42e1-8460-5dcf6b57b98c"}},{"where":[["_uuid","==",["uuid","7145cd6b-c2a5-4cb7-93fa-af8e1e3e009e"]]],"table":"Logical_Switch_Port","op":"update","row":{"addresses":"router","options":["map",[["nat-addresses","router"],["router-port","lrp-488b3887-1770-4a72-86cb-7306e78c954a"]]],"type":"router"}},{"uuid-name":"row9d1df734_4...

Read more...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers