centos-8 train - fs35 job fails on various tempest tests

[Soniya Murlidhar Vyas]

periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train job fails on various tempest tests in different job runs.

Following is the traceback of latest run:-

1. setUpClass (tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON)

ft1.1: setUpClass (tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON)testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 372, in create_image_from_server
    waiters.wait_for_image_status(client, image_id, wait_until)
  File "/usr/lib/python3.6/site-packages/tempest/common/waiters.py", line 167, in wait_for_image_status
    image = show_image(image_id)
  File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/images_client.py", line 75, in show_image
    resp, body = self.get("images/%s" % image_id)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 315, in get
    return self.request('GET', url, extra_headers, headers)
  File "/usr/lib/python3.6/site-packages/tempest/lib/services/compute/base_compute_client.py", line 48, in request
    method, url, extra_headers, headers, body, chunked)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 704, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 810, in _error_checker
    raise exceptions.NotFound(resp_body, resp=resp)
tempest.lib.exceptions.NotFound: Object not found
Details: {'code': 404, 'message': 'Image not found.'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/test.py", line 182, in setUpClass
    six.reraise(etype, value, trace)
  File "/usr/local/lib/python3.6/site-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/tempest/test.py", line 175, in setUpClass
    cls.resource_setup()
  File "/usr/lib/python3.6/site-packages/tempest/api/compute/images/test_list_image_filters.py", line 124, in resource_setup
    cls.server2['id'], wait_until='ACTIVE')
  File "/usr/lib/python3.6/site-packages/tempest/api/compute/base.py", line 387, in create_image_from_server
    image_id=image_id)
tempest.exceptions.SnapshotNotFoundException: Server snapshot image b4edeb9e-113a-447d-a69b-e0a996ec1f41 not found.

For more details please refer following logs:
- https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/7101aee/logs/undercloud/var/log/tempest/stestr_results.html.gz
- https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/cd8b9ab/logs/undercloud/var/log/tempest/stestr_results.html.gz
- https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/ff24b1d/logs/undercloud/var/log/tempest/stestr_results.html.gz

[Amol Kahat (amolkahat)]

Log:
====
- https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/7101aee/logs/undercloud/var/log/containers/nova/nova-scheduler.log.txt.gz

Warning:
========
2021-11-15 20:27:29.825 7 WARNING nova.scheduler.client.report [req-d53bbe6f-7e24-45a1-b680-1aa67d6f0560 596049df6a1b43e2b4cd0e6fbd111787 8e2869f438d74ce384f197d15063a132 - default default] Failed to save allocation for 3e2eda9f-e806-4667-aa8c-0a16250dde7f. Got HTTP 409: {"errors": [{"status": 409, "title": "Conflict", "detail": "There was a conflict when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'CUSTOM_BAREMETAL' on resource provider 'b9853310-bbc0-427e-a0a1-71c361eae0a1'. The requested amount would exceed the capacity. ", "code": "placement.undefined_code", "request_id": "req-22c5299c-56da-4477-a28a-71dc1dd701e7"}]}

[Ronelle Landy (rlandy)]

https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train&pipeline=openstack-periodic-integration-stable4
 looks to be working now