MessagingTimeout errors in unit tests
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Low
|
Unassigned |
Bug Description
These can be seen all over the unit test logs. At least some of them are caused by tests failing to mock calls to conductor api method build_instances(), which is spawning new threads to handle such builds. The timeouts happen when calls to scheduler gets no reply within the configured rpc timeout of 60 secs.
This is not actually causing any test failures but makes debugging harder since errors show up randomly in logs.
A typical error looks like this:
Traceback (most recent call last):
File "nova/conductor
request_spec, filter_properties)
File "nova/scheduler
context, request_spec, filter_properties)
File "nova/scheduler
return getattr(
File "nova/scheduler
context, request_spec, filter_properties)
File "nova/scheduler
request_
File "/home/
retry=
File "/home/
timeout=
File "/home/
return self._send(target, ctxt, message, wait_for_reply, timeout)
File "/home/
'No reply on topic %s' % target.topic)
MessagingTimeout: No reply on topic scheduler
WARNING [nova.scheduler
Then followed by an attempt to set the instance to ERROR state, which fails since the instance does not exist in the database.
Traceback (most recent call last):
File "/home/
timer()
File "/home/
cb(*args, **kw)
File "nova/utils.py", line 949, in wrapper
return func(*args, **kwargs)
File "nova/conductor
instance.uuid, request_spec)
File "nova/scheduler
'task_state': None})
File "nova/db/api.py", line 746, in instance_
columns_
File "nova/db/
return f(*args, **kwargs)
File "nova/db/
columns_
File "nova/db/
columns_
File "nova/db/
raise exception.
description: | updated |
Changed in nova: | |
importance: | Undecided → Low |
Changed in nova: | |
assignee: | Hans Lindgren (hanlind) → Mike Durnosvistov (mdurnosvistov) |
Fix proposed to branch: master /review. openstack. org/122726
Review: https:/