ovsdbapp

  1. Bug #1965819
  2. Comment #12

Comment 12 for bug 1965819

Revision history for this message
Frode Nordahl (fnordahl) wrote :

There appears to be more to it. With this patch applied Neutron will still fail to start with the OVN driver when OVS 2.17 is installed.

With this instrumentation of db/idl.py:
diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py
index 4ecdcaa19..d676a39b3 100644
--- a/python/ovs/db/idl.py
+++ b/python/ovs/db/idl.py
@@ -357,6 +357,7 @@ class Idl(object):
         """
         ack_all = self.last_id == str(uuid.UUID(int=0))
         for table in self.tables.values():
+ logging.info('HELLO ovs.db {}'.format(table.condition))
             if ack_all:
                 table.condition.request()
                 table.condition.ack()

I see:
2022-04-04 16:32:29.270 103953 INFO neutron.service [req-07913206-4206-49fc-bbfc-710c0e8f992a - - - - -] Neutron service started, listening on 0.0.0.0:9676
2022-04-04 16:32:29.296 103961 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-d6dc8c25-6296-43e4-9858-a93274def15d - - - - -] Getting OvsdbNbOvnIdl for WorkerService with retry
2022-04-04 16:32:29.297 103962 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-9d84dcb7-7b4f-4fad-8f87-c6b5238f7630 - - - - -] Getting OvsdbNbOvnIdl for WorkerService with retry
2022-04-04 16:32:29.297 103963 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-3fa2a398-db18-4319-9f3d-6067b55d1699 - - - - -] Getting OvsdbNbOvnIdl for WorkerService with retry
2022-04-04 16:32:29.299 103964 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-2821d81a-304f-491e-a053-ba845f5925cf - - - - -] Getting OvsdbNbOvnIdl for WorkerService with retry
2022-04-04 16:32:29.306 103953 INFO root [req-07913206-4206-49fc-bbfc-710c0e8f992a - - - - -] HELLO ovs.db [['name', '==', 'neutron_pg_drop']]
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-07913206-4206-49fc-bbfc-710c0e8f992a - - - - -] OVS database connection to OVN_Northbound failed with error: ''list' object has no attribute 'request''. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: AttributeError: 'list' object has no attribute 'request'
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 127, in start_connection
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.ovsdb_connection.start()
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 83, in start
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn idlutils.wait_for_change(self.idl, self.timeout)
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 53, in wait_for_change
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn while idl_.change_seqno == seqno and not idl_.run():
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 412, in run
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.restart_fsm()
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 374, in restart_fsm
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn self.sync_conditions()
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 363, in sync_conditions
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn table.condition.request()
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn AttributeError: 'list' object has no attribute 'request'
2022-04-04 16:32:29.306 103953 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
2022-04-04 16:32:29.308 103953 CRITICAL neutron [req-07913206-4206-49fc-bbfc-710c0e8f992a - - - - -] Unhandled error: neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-198156 failed with "OVS database connection to OVN_Northbound failed with error: ''list' object has no attribute 'request''. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct."
2022-04-04 16:32:29.308 103953 ERROR neutron Traceback (most recent call last):
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/bin/neutron-server", line 10, in <module>
2022-04-04 16:32:29.308 103953 ERROR neutron sys.exit(main())
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/server/__init__.py", line 19, in main
2022-04-04 16:32:29.308 103953 ERROR neutron server.boot_server(wsgi_eventlet.eventlet_wsgi_server)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/__init__.py", line 68, in boot_server
2022-04-04 16:32:29.308 103953 ERROR neutron server_func()
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/wsgi_eventlet.py", line 24, in eventlet_wsgi_server
2022-04-04 16:32:29.308 103953 ERROR neutron neutron_api = service.serve_wsgi(service.NeutronApiService)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 94, in serve_wsgi
2022-04-04 16:32:29.308 103953 ERROR neutron registry.publish(resources.PROCESS, events.BEFORE_SPAWN, service)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/registry.py", line 54, in publish
2022-04-04 16:32:29.308 103953 ERROR neutron _get_callback_manager().publish(resource, event, trigger, payload=payload)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 110, in _wrapped
2022-04-04 16:32:29.308 103953 ERROR neutron raise db_exc.RetryRequest(e)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
2022-04-04 16:32:29.308 103953 ERROR neutron self.force_reraise()
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
2022-04-04 16:32:29.308 103953 ERROR neutron raise self.value
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 105, in _wrapped
2022-04-04 16:32:29.308 103953 ERROR neutron return function(*args, **kwargs)
2022-04-04 16:32:29.308 103953 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 157, in publish
2022-04-04 16:32:29.308 103953 ERROR neutron raise exceptions.CallbackFailure(errors=errors)
2022-04-04 16:32:29.308 103953 ERROR neutron neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-198156 failed with "OVS database connection to OVN_Northbound failed with error: ''list' object has no attribute 'request''. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct."
2022-04-04 16:32:29.308 103953 ERROR neutron

The condition appear appear to come from Neutron somehow:
https://github.com/openstack/neutron/blob/3615cd85a4cc6aeecf7f066c4eb21c3cdca71d4c/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L660

So I guess the question then becomes; does the change to the OVS Python IDL need more guardrails, or does consumers need to be updated before they can be used with the OVS 2.17 Python IDL?