OpenStack Nova Cloud Controller Charm

  1. Bug #1861094
  2. Comment #4

Comment 4 for bug 1861094

Revision history for this message
Aurelien Lourot (aurelien-lourot) wrote :

Happened here right after a deployment on focal: https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_func_full/openstack/charm-mysql-router/748714/4/6723/index.html

2020-08-29 10:08:56.024 95537 ERROR oslo_service.service [req-9ecebbf6-d8b1-4304-9258-f6e9e0c9b6a2 - - - - -] Error starting thread.: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting fo
r a reply to message ID 1491906c55764d4fbd478c75640fa0aa
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service Traceback (most recent call last):
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 405, in get
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return self._queues[msg_id].get(block=True, timeout=timeout)
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 322, in get
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return waiter.wait()
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 141, in wait
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return get_hub().switch()
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 298, in switch
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return self.greenlet.switch()
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service _queue.Empty
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service During handling of the above exception, another exception occurred:
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service Traceback (most recent call last):
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 810, in run_service
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service service.start()
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/nova/service.py", line 172, in start
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service self.manager.init_host()
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1399, in init_host
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service instances = objects.InstanceList.get_by_host(
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_versionedobjects/base.py", line 175, in wrapper
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service result = cls.indirection_api.object_class_action_versions(
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/nova/conductor/rpcapi.py", line 240, in object_class_action_versions
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return cctxt.call(context, 'object_class_action_versions',
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 177, in call
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service self.transport._send(self.target, msg_ctxt, msg,
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 124, in _send
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return self._driver.send(target, ctxt, message,
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in send
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service return self._send(target, ctxt, message, wait_for_reply, timeout,
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 641, in _send
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service result = self._waiter.wait(msg_id, timeout,
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 531, in wait
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service message = self.waiters.get(msg_id, timeout=timeout)
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 407, in get
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service raise oslo_messaging.MessagingTimeout(
2020-08-29 10:08:56.024 95537 ERROR oslo_service.service oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 1491906c55764d4fbd478c75640fa0aa

OSCI is quite busy at the moment, so it might be caused by a very loaded undercloud.