tripleo

OVB: Nova compute fails with error BadRequestException: 400: Client Error for url: http://192.168.24.3:6385/v1/nodes?fields=instance_id&associated=True, Field(s) "instance_id" are not valid

Bug #1842933 reported by Sagi (Sergey) Shnaidman on 2019-09-05

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	Critical	Unassigned	tripleo train-rc1

Bug Description

Nova moved to openstacksdk and seems like it broke OVB jobs.

(patch of Nova: https://review.opendev.org/#/c/656027/)

Failures:

http://logs.rdoproject.org/98/22098/1/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/96a052c/logs/undercloud/var/log/extra/errors.txt.txt.gz

2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task Traceback (most recent call last):
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 222, in run_periodic_tasks
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task task(self, context)
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8400, in _cleanup_running_deleted_instances
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task for instance in self._running_deleted_instances(context):
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8454, in _running_deleted_instances
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task instances = self._get_instances_on_driver(context, filters)
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 626, in _get_instances_on_driver
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task driver_uuids = self.driver.list_instance_uuids()
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 705, in list_instance_uuids
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task return_generator=True, associated=True, fields=['instance_id'])]
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/openstack/resource.py", line 1588, in list
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task exceptions.raise_from_response(response)
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/openstack/exceptions.py", line 229, in raise_from_response
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task http_status=http_status, request_id=request_id
2019-09-05 10:36:09.502 ERROR /var/log/containers/nova/nova-compute.log: 9 ERROR oslo_service.periodic_task BadRequestException: BadRequestException: 400: Client Error for url: http://192.168.24.3:6385/v1/nodes?fields=instance_id&associated=True, Field(s) "instance_id" are not valid

And then Ironic fails to power on:

2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last):
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/agent_base_vendor.py", line 669, in reboot_and_finish_deploy
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor _wait_until_powered_off(task)
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor return Retrying(*dargs, **dkw).call(f, *args, **kw)
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor raise RetryError(attempt)
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor RetryError: RetryError[Attempts: 7, Value: power on]
2019-09-05 09:38:32.034 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last):
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/agent_base_vendor.py", line 669, in reboot_and_finish_deploy
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor _wait_until_powered_off(task)
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor return Retrying(*dargs, **dkw).call(f, *args, **kw)
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor raise RetryError(attempt)
2019-09-05 09:44:33.378 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.drivers.modules.agent_base_vendor RetryError: RetryError[Attempts: 7, Value: power on]