openstack server list error

Bug #1789382 reported by cc
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Expired
Undecided
Unassigned

Bug Description

when execute openstack server list,Show the following error:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-6a17a629-6bc5-4297-90b4-524dd96b5694)
everytime I see this in nova-api.log:

2018-08-28 16:01:12.660 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 4d2b47d3-7d78-4061-97fe-89c32e878fa7: CellTimeout: Timeout waiting for response from cell
2018-08-28 16:01:12.661 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 00000000-0000-0000-0000-000000000000: CellTimeout: Timeout waiting for response from cell
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Unexpected exception in API method: TypeError: 'object' object is not iterable
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 165, in detail
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi servers = self._get_servers(req, is_detail=True)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 309, in _get_servers
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2500, in get_all
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi context, filters, limit, marker, fields, sort_keys, sort_dirs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 107, in get_instance_objects_sorted
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi expected_attrs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1208, in _make_instance_list
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi for db_inst in db_inst_list:
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 259, in get_records_sorted
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi for i in heapq.merge(*results.values()):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/heapq.py", line 372, in merge
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi for itnum, it in enumerate(map(iter, iterables)):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi TypeError: 'object' object is not iterable
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi
2018-08-28 16:01:12.702 1092436 INFO nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'>
2018-08-28 16:01:12.705 1092436 INFO nova.osapi_compute.wsgi.server [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] 18.5.253.11 "GET /v2.1/servers/detail HTTP/1.1" status: 500 len: 626 time: 120.9522789

When I log in to the dashboard, I have to wait a long time and report the same error.
I have checked the database configuration of nova.conf。

if anyone have or had this problem and can propose a fix i would appreciate it, thanks in advance.
Note: i could provide anything if needed

Tags: api cells
tags: added: api cells
Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

it seems this is hitting in [1] and i suspect object returned in None which leads to 500.

Can you please provide the compute api full logs for further debugging.

[1]
https://github.com/openstack/nova/blob/c6218428e9b29a2c52808ec7d27b4b21aadc0299/nova/comp
ute/multi_cell_list.py#L265

Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

marking as incomplete till log are available.

Changed in nova:
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]

Changed in nova:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.