Comment 16 for bug 1844929

Revision history for this message
Matt Riedemann (mriedem) wrote :

I just noticed this in a nova functional job as well:

https://zuul.opendev.org/t/openstack/build/63001bbd58c244cea70c995f1ebf61fb/log/job-output.txt#3092

In this case it looks like a result of a slow node:

2019-11-05 23:12:57.389624 | ubuntu-bionic | 2019-11-05 23:10:12,022 INFO [nova.scheduler.host_manager] Received an update from an unknown host 'host2'. Re-created its InstanceList.

2019-11-05 23:12:57.389814 | ubuntu-bionic | 2019-11-05 23:10:12,569 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/servers/9ceb2e4d-9bb3-425b-9454-b171221d6c93" status: 200 len: 2019 microversion: 2.15 time: 0.048796

2019-11-05 23:12:57.389957 | ubuntu-bionic | 2019-11-05 23:11:23,400 WARNING [oslo.service.loopingcall] Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run outlasted interval by 59.77 sec

2019-11-05 23:12:57.390092 | ubuntu-bionic | 2019-11-05 23:11:23,414 WARNING [nova.context] Timed out waiting for response from cell 3d90ac16-9173-4df8-8ef9-1d713ccd8a98

2019-11-05 23:12:57.390189 | ubuntu-bionic | 2019-11-05 23:11:23,415 ERROR [nova.api.openstack.wsgi] Unexpected exception in API method

2019-11-05 23:12:57.390236 | ubuntu-bionic | Traceback (most recent call last):

2019-11-05 23:12:57.390299 | ubuntu-bionic | File "nova/api/openstack/wsgi.py", line 671, in wrapped

2019-11-05 23:12:57.390340 | ubuntu-bionic | return f(*args, **kwargs)

2019-11-05 23:12:57.390406 | ubuntu-bionic | File "nova/api/validation/__init__.py", line 110, in wrapper

2019-11-05 23:12:57.390450 | ubuntu-bionic | return func(*args, **kwargs)

2019-11-05 23:12:57.390515 | ubuntu-bionic | File "nova/api/validation/__init__.py", line 110, in wrapper

2019-11-05 23:12:57.390559 | ubuntu-bionic | return func(*args, **kwargs)

2019-11-05 23:12:57.390625 | ubuntu-bionic | File "nova/api/validation/__init__.py", line 110, in wrapper

2019-11-05 23:12:57.390668 | ubuntu-bionic | return func(*args, **kwargs)

2019-11-05 23:12:57.390734 | ubuntu-bionic | File "nova/api/validation/__init__.py", line 110, in wrapper

2019-11-05 23:12:57.390778 | ubuntu-bionic | return func(*args, **kwargs)

2019-11-05 23:12:57.390850 | ubuntu-bionic | File "nova/api/openstack/compute/evacuate.py", line 85, in _evacuate

2019-11-05 23:12:57.390918 | ubuntu-bionic | instance = common.get_instance(self.compute_api, context, id)

2019-11-05 23:12:57.390986 | ubuntu-bionic | File "nova/api/openstack/common.py", line 472, in get_instance

2019-11-05 23:12:57.391035 | ubuntu-bionic | cell_down_support=cell_down_support)

2019-11-05 23:12:57.391104 | ubuntu-bionic | File "nova/compute/api.py", line 2722, in get

2019-11-05 23:12:57.391173 | ubuntu-bionic | expected_attrs, cell_down_support=cell_down_support)

2019-11-05 23:12:57.391236 | ubuntu-bionic | File "nova/compute/api.py", line 2669, in _get_instance

2019-11-05 23:12:57.391284 | ubuntu-bionic | expected_attrs, cell_down_support)

2019-11-05 23:12:57.391354 | ubuntu-bionic | File "nova/compute/api.py", line 2662, in _get_instance_from_cell

2019-11-05 23:12:57.391405 | ubuntu-bionic | "info is not available.") % cell_uuid)

2019-11-05 23:12:57.391514 | ubuntu-bionic | NovaException: Cell 3d90ac16-9173-4df8-8ef9-1d713ccd8a98 is not responding and hence instance info is not available.

2019-11-05 23:12:57.391704 | ubuntu-bionic | 2019-11-05 23:11:23,418 INFO [nova.api.openstack.wsgi] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

2019-11-05 23:12:57.391769 | ubuntu-bionic | <class 'nova.exception.NovaException'>