| 2020-11-26 09:54:31 |
Lucas Alvares Gomes |
description |
Depending on the order that things are started, the ovn-metadata-agent may fail at start up with the following error [0].
This seems to happen because the Chassis is not yet registered (by ovn-controller) in the OVSDB SB database.
I noticed this is more likely to happen when stacking DevStack and services are started one after the other. Perhaps the ovn-metadata-agent should retry fetching it's own chassis from the OVSDB SB database a few times before failing to start as you can see on other logs that the Chassis will appear a few seconds later.
[0]
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron Traceback (most recent call last):
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/bin/neutron-ovn-metadata-agent", line 10, in <module>
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron sys.exit(main())
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/cmd/eventlet/agents/ovn_metadata.py", line 24, in main
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron metadata_agent.main()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata_agent.py", line 39, in main
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron agt.start()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata/agent.py", line 247, in start
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.register_metadata_agent()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata/agent.py", line 257, in register_metadata_agent
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.sb_idl.db_add(table, self.chassis, 'external_ids',
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron t.add(self)
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron next(self.gen)
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/api.py", line 110, in transaction
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron del self._nested_txns_map[cur_thread_id]
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/api.py", line 61, in __exit__
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.result = self.commit()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 63, in commit
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron raise result.ex
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 129, in run
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron txn.results.put(txn.do_commit())
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 90, in do_commit
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron command.run_idl(txn)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 171, in run_idl
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron record = self.api.lookup(self.table, self.record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 177, in lookup
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron return self._lookup(table, record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 224, in _lookup
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron row = idlutils.row_by_value(self, rl.table, rl.column, record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron raise RowNotFound(table=table, col=column, match=match)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Chassis with name=fb530244-3537-4efa-ab6a-7c5c9683c38c
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron |
Depending on the order that things are started, the ovn-metadata-agent may fail at start up with the following error [0].
This seems to happen because the Chassis is not yet registered (by ovn-controller) in the OVSDB SB database.
I noticed this is more like to happen when stacking DevStack and services are started one after the other. Perhaps the ovn-metadata-agent should retry fetching it's own chassis from the OVSDB SB database a few times before failing to start as you can see on other logs that the Chassis will appear a few seconds later.
[0]
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron Traceback (most recent call last):
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/bin/neutron-ovn-metadata-agent", line 10, in <module>
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron sys.exit(main())
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/cmd/eventlet/agents/ovn_metadata.py", line 24, in main
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron metadata_agent.main()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata_agent.py", line 39, in main
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron agt.start()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata/agent.py", line 247, in start
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.register_metadata_agent()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/opt/stack/neutron/neutron/agent/ovn/metadata/agent.py", line 257, in register_metadata_agent
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.sb_idl.db_add(table, self.chassis, 'external_ids',
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron t.add(self)
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron next(self.gen)
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/api.py", line 110, in transaction
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron del self._nested_txns_map[cur_thread_id]
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/api.py", line 61, in __exit__
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron self.result = self.commit()
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 63, in commit
Nov 25 19:34:45.455716 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron raise result.ex
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 129, in run
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron txn.results.put(txn.do_commit())
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 90, in do_commit
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron command.run_idl(txn)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 171, in run_idl
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron record = self.api.lookup(self.table, self.record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 177, in lookup
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron return self._lookup(table, record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 224, in _lookup
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron row = idlutils.row_by_value(self, rl.table, rl.column, record)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron File "/usr/local/lib/python3.8/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron raise RowNotFound(table=table, col=column, match=match)
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Chassis with name=fb530244-3537-4efa-ab6a-7c5c9683c38c
Nov 25 19:34:45.457508 ubuntu-focal-limestone-regionone-0021931462 neutron-ovn-metadata-agent[66875]: ERROR neutron |
|
