Yeah the problem is cell0:
http://logs.openstack.org/05/407205/1/check/gate-novaclient-dsvm-functional-identity-v3-only-ubuntu-xenial-nv/61e9a05/logs/screen-n-api.txt.gz#_2016-12-06_20_58_54_417
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api [req-de260414-aacf-47c4-949b-0b501efa5e69 admin admin] Failed to find instance by marker fe05d5ed-e97a-48c3-a57f-fb8a05f43d88 in cell0; ignoring 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api Traceback (most recent call last): 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/compute/api.py", line 2404, in get_all 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_dirs=sort_dirs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/compute/api.py", line 2483, in _get_instances_by_filters 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api result = fn(cls, context, *args, **kwargs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/objects/instance.py", line 1219, in get_by_filters 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api use_slave=use_slave, sort_keys=sort_keys, sort_dirs=sort_dirs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 226, in wrapper 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(*args, **kwargs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/objects/instance.py", line 1203, in _get_by_filters_impl 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_keys=sort_keys, sort_dirs=sort_dirs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/db/api.py", line 763, in instance_get_all_by_filters_sort 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_dirs=sort_dirs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 170, in wrapper 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(*args, **kwargs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 271, in wrapped 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(context, *args, **kwargs) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 2243, in instance_get_all_by_filters_sort 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api raise exception.MarkerNotFound(marker=marker) 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api MarkerNotFound: Marker fe05d5ed-e97a-48c3-a57f-fb8a05f43d88 could not be found. 2016-12-06 20:58:54.417 27543 ERROR nova.compute.api
So we just need to handle that MarkerNotFound and ignore it when listing instances.
Yeah the problem is cell0:
http:// logs.openstack. org/05/ 407205/ 1/check/ gate-novaclient -dsvm-functiona l-identity- v3-only- ubuntu- xenial- nv/61e9a05/ logs/screen- n-api.txt. gz#_2016- 12-06_20_ 58_54_417
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api [req-de260414- aacf-47c4- 949b-0b501efa5e 69 admin admin] Failed to find instance by marker fe05d5ed- e97a-48c3- a57f-fb8a05f43d 88 in cell0; ignoring new/nova/ nova/compute/ api.py" , line 2404, in get_all sort_dirs) new/nova/ nova/compute/ api.py" , line 2483, in _get_instances_ by_filters attrs=fields, sort_keys= sort_keys, sort_dirs= sort_dirs) lib/python2. 7/dist- packages/ oslo_versionedo bjects/ base.py" , line 184, in wrapper new/nova/ nova/objects/ instance. py", line 1219, in get_by_filters use_slave, sort_keys= sort_keys, sort_dirs= sort_dirs) new/nova/ nova/db/ sqlalchemy/ api.py" , line 226, in wrapper new/nova/ nova/objects/ instance. py", line 1203, in _get_by_ filters_ impl sort_keys, sort_dirs= sort_dirs) new/nova/ nova/db/ api.py" , line 763, in instance_ get_all_ by_filters_ sort sort_dirs) new/nova/ nova/db/ sqlalchemy/ api.py" , line 170, in wrapper new/nova/ nova/db/ sqlalchemy/ api.py" , line 271, in wrapped new/nova/ nova/db/ sqlalchemy/ api.py" , line 2243, in instance_ get_all_ by_filters_ sort MarkerNotFound( marker= marker) e97a-48c3- a57f-fb8a05f43d 88 could not be found.
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api Traceback (most recent call last):
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_dirs=
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api expected_
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/usr/local/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api result = fn(cls, context, *args, **kwargs)
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api use_slave=
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(*args, **kwargs)
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_keys=
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api sort_dirs=
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(*args, **kwargs)
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api return f(context, *args, **kwargs)
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api File "/opt/stack/
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api raise exception.
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api MarkerNotFound: Marker fe05d5ed-
2016-12-06 20:58:54.417 27543 ERROR nova.compute.api
So we just need to handle that MarkerNotFound and ignore it when listing instances.