[fullstack] OVS interface events isolation error with more than one OVS agent

Bug #1885547 reported by Rodolfo Alonso
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Low
Rodolfo Alonso

Bug Description

In fullstack test, unlike in a real deployment, more than one OVS agent is spawned in the same host, reading from the same OVS database. That could lead to isolation errors when receiving notifications. For example, OVS has a thread monitoring the new ports added to the host, using the shell CLI:
  "ovsdb-client monitor Interface name,ofport,external_ids"

In a fullstack test, two OVS agents running in the same host will have independent integration bridges (randomly generated names). But when a port is added to one of those integration bridges, both agents will receive the event of a new port (interface) added. The OVS agent where the port is not added, won't find it in its own integration bridge and will treat it as dead port [1], sending a "update_device_down" message to the server.

This behavior could clash with the correct update sent from the other OVS agent, setting the port up. E.g.: in log [2] we can see how the first OVS agent sets the port UP and one second later, the second OVS agent sets the port down [3]. That will make this test fail.

[1]https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1833-L1834
[2]https://4317fd6ef51d22a4668e-8daa7927094e2ef08ab8f0f95a49fc40.ssl.cf5.rackcdn.com/733250/18/check/neutron-fullstack-with-uwsgi/79ec86c/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_external_subnet_changed/neutron-server--2020-06-26--12-24-50-479143_log.txt
[3]http://paste.openstack.org/show/795317/

Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/738446

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → Low
tags: added: fullstack gate-failure ovs
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.0.0.0rc1

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.

Changed in neutron:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.