nova.tests.functional.libvirt.test_pci_sriov_servers.VDPAServersTest.test_suspend_and_resume failed for one of my patches. Failure itself looks sporadic, but may be a sign of underlying problems.
Complete log: https://3597fc80eb5e70afafd6-8fe1ce567603c46a794844f7885416e9.ssl.cf1.rackcdn.com/873388/2/check/nova-tox-functional-py310/6921a72/job-output.txt
My understanding:
- "Original exception" [1] is logged when nova-conductor is started
- This exception triggers calls, which in turn cause libvirt import and trigger ImportError exception:
2023-02-17 11:43:02.536761 | ubuntu-jammy | 2023-02-17 11:26:59,887 WARNING [nova.tests.fixtures.nova] !!!---!!! TestCase ID nova.tests.functional.libvirt.test_pci_sriov_servers.VDPAServersTest.test_suspend_and_resume hit the import poison while importing libvirt. If you see this in a failed functional test then please let #openstack-nova on IRC know about it. !!!---!!!
It looks like original issue is caused by some RPC call related to live migration.
This problem seem to be sporadic and is not a blocker, but it looks quite interesting and could affect CI in general, so I am reporting this bug.
[1]
2023-02-17 11:43:02.529191 | ubuntu-jammy | 2023-02-17 11:26:59,782 ERROR [root] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/oslo_messaging/_drivers/impl_fake.py", line 207, in _send\n reply, failure = reply_q.get(timeout=timeout)\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/eventlet/queue.py", line 322, in get\n return waiter.wait()\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/eventlet/queue.py", line 141, in wait\n return get_hub().switch()\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', '_queue.Empty\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 203, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 9306, in _post_live_migration\n self._update_scheduler_instance_info(ctxt, instance)\n', ' File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 2225, in _update_scheduler_instance_info\n self.query_client.update_instance_info(context, self.host,\n', ' File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 69, in update_instance_info\n self.scheduler_rpcapi.update_instance_info(context, host_name,\n', ' File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 174, in update_instance_info\n cctxt.cast(ctxt, \'update_instance_info\', host_name=host_name,\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/fixtures/_fixtures/monkeypatch.py", line 86, in avoid_get\n return captured_method(*args, **kwargs)\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in c
2023-02-17 11:43:02.529623 | ubuntu-jammy | all\n result = self.transport._send(\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send\n return self._driver.send(target, ctxt, message,\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send\n return self._send(target, ctxt, message, wait_for_reply, timeout,\n', ' File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py310/lib/python3.10/site-packages/oslo_messaging/_drivers/impl_fake.py", line 213, in _send\n raise oslo_messaging.MessagingTimeout(\n', 'oslo_messaging.exceptions.MessagingTimeout: No reply on topic scheduler\n']
2023-02-17 11:43:02.529678 | ubuntu-jammy | 2023-02-17 11:26:59,881 WARNING [nova.virt.libvirt.driver] Error monitoring migration: (sqlite3.OperationalError) no such table: compute_nodes
Thanks for reporting this bug but it's a duplicate of https:/
We wrote a patch for trying to understand which test was calling a specific thread.