I've opened it up initially with openvswitch team before I dug a bit more into it. It looks like changes in 2.17 code introduce condition attribute to sync changes. However, neutron also use condition attribute, e.g. neutron/agent/ovn/metadata/ovsdb.py
class MetadataAgentOvnSbIdl(ovsdb_monitor.OvnIdl):
...
if chassis:
for table in set(tables).intersection({'Chassis',
'Chassis_Private'}):
self.tables[table].condition = [['name', '==', chassis]]
So what happen is ovs idl code is falling with the following error:
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] OVS database connection to OVN_Southbound failed with error: ''list' object has no attribute 'request''. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: AttributeError: 'list' object has no attribute 'request'
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 127, in start_connection
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.ovsdb_connection.start()
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 83, in start
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn idlutils.wait_for_change(self.idl, self.timeout)
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 53, in wait_for_change
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn while idl_.change_seqno == seqno and not idl_.run():
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 421, in run
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.restart_fsm()
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 383, in restart_fsm
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.sync_conditions()
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 372, in sync_conditions
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn table.condition.request()
2022-03-18 21:55:11.894 24 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn AttributeError: 'list' object has no attribute 'request'
When I've added a few print lines to find out what is going on, I found out that ovs is expecting ConditionState object, but neutron code is also using condition attribute and when I've override __setattr__ func to find out who is overwrite the ovs code. It turns out ovsdb.py also use that attribute.
File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/callbacks/manager.py", line 150, in publish
errors = self._notify_loop(resource, event, trigger, payload)
File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/callbacks/manager.py", line 181, in _notify_loop
callback(resource, event, trigger, payload=payload)
File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/agent/ovn/metadata/server.py", line 74, in post_fork_initialize
self.sb_idl = ovsdb.MetadataAgentOvnSbIdl(
File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/agent/ovn/metadata/ovsdb.py", line 53, in __init__
self.tables[table].condition = [['name', '==', chassis]]
File "/usr/lib/python3/dist-packages/ovs/db/schema.py", line 190, in __setattr__
vlog.err(''.join(traceback.format_stack()))
https://opendev.org/openstack/neutron/src/branch/master/neutron/agent/ovn/metadata/ovsdb.py
class MetadataAgentOvnSbIdl(ovsdb_monitor.OvnIdl):
SCHEMA = 'OVN_Southbound'
def __init__(self, chassis=None, events=None, tables=None):
connection_string = config.get_ovn_sb_connection()
ovsdb_monitor._check_and_set_ssl_files(self.SCHEMA)
helper = self._get_ovsdb_helper(connection_string)
if tables is None:
tables = ('Chassis', 'Encap', 'Port_Binding', 'Datapath_Binding',
'SB_Global')
for table in tables:
helper.register_table(table)
try:
super(MetadataAgentOvnSbIdl, self).__init__(
None, connection_string, helper, leader_only=False)
except TypeError:
# TODO(twilson) We can remove this when we require ovs>=2.12.0
super(MetadataAgentOvnSbIdl, self).__init__(
None, connection_string, helper)
if chassis:
for table in set(tables).intersection({'Chassis',
'Chassis_Private'}):
self.tables[table].condition = [['name', '==', chassis]]
The workaround at the moment is to downgrade to 2.16, however, it's problematic as neutron requiments.txt are only setting lower limit for ovs package.
ovs>=2.10.0 # Apache-2.0
Small comment, I've hit the bug when building my OpenStack using the Kolla master branch which has been set for Yoga and used the Ubuntu cloud package for Yoga which is 2.17. It's not an issue for Xena.