CI job 'grenade-py3' broken

[Eric Xie]

The log as below:

020-03-05 15:42:17.659948 | primary | Captured traceback:
2020-03-05 15:42:17.659954 | primary | ~~~~~~~~~~~~~~~~~~~
2020-03-05 15:42:17.659961 | primary | Traceback (most recent call last):
2020-03-05 15:42:17.659967 | primary | File "/opt/stack/new/tempest/tempest/test.py", line 188, in setUpClass
2020-03-05 15:42:17.659974 | primary | six.reraise(etype, value, trace)
2020-03-05 15:42:17.659980 | primary | File "/opt/stack/new/tempest/.tox/tempest/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-03-05 15:42:17.659986 | primary | raise value
2020-03-05 15:42:17.659993 | primary | File "/opt/stack/new/tempest/tempest/test.py", line 181, in setUpClass
2020-03-05 15:42:17.659999 | primary | cls.resource_setup()
2020-03-05 15:42:17.660010 | primary | File "/opt/stack/new/tempest/tempest/api/compute/servers/test_server_actions.py", line 81, in resource_setup
2020-03-05 15:42:17.660017 | primary | cls.server_id = cls.recreate_server(None, validatable=True)
2020-03-05 15:42:17.660023 | primary | File "/opt/stack/new/tempest/tempest/api/compute/base.py", line 424, in recreate_server
2020-03-05 15:42:17.660030 | primary | **kwargs)
2020-03-05 15:42:17.660036 | primary | File "/opt/stack/new/tempest/tempest/api/compute/base.py", line 263, in create_test_server
2020-03-05 15:42:17.660042 | primary | **kwargs)
2020-03-05 15:42:17.660049 | primary | File "/opt/stack/new/tempest/tempest/common/compute.py", line 271, in create_test_server
2020-03-05 15:42:17.660055 | primary | server['id'])
2020-03-05 15:42:17.660062 | primary | File "/opt/stack/new/tempest/.tox/tempest/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-03-05 15:42:17.660068 | primary | self.force_reraise()
2020-03-05 15:42:17.660074 | primary | File "/opt/stack/new/tempest/.tox/tempest/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-03-05 15:42:17.660081 | primary | six.reraise(self.type_, self.value, self.tb)
2020-03-05 15:42:17.660087 | primary | File "/opt/stack/new/tempest/.tox/tempest/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-03-05 15:42:17.660094 | primary | raise value
2020-03-05 15:42:17.660100 | primary | File "/opt/stack/new/tempest/tempest/common/compute.py", line 242, in create_test_server
2020-03-05 15:42:17.660108 | primary | clients.servers_client, server['id'], wait_until)
2020-03-05 15:42:17.660121 | primary | File "/opt/stack/new/tempest/tempest/common/waiters.py", line 76, in wait_for_server_status
2020-03-05 15:42:17.660133 | primary | server_id=server_id)
2020-03-05 15:42:17.660146 | primary | tempest.exceptions.BuildErrorException: Server fc549fbb-8344-4b15-af35-6e7662270dd0 failed to build and is in ERROR status
2020-03-05 15:42:17.660154 | primary | Details: {'code': 500, 'created': '2020-03-05T15:38:04Z', 'message': 'No valid host was found. There are not enough hosts available.'}

[Martin Kopec (mkopec)]

Is this related to tempest? shouldn't this be opened against granade or infra?

[Paras Babbar (pbabbar)]

Hi,

This seems like an Infra issue which can be occur when the server was not able to build:

2020-03-05 15:42:17.660146 | primary | tempest.exceptions.BuildErrorException: Server fc549fbb-8344-4b15-af35-6e7662270dd0 failed to build and is in ERROR status
2020-03-05 15:42:17.660154 | primary | Details: {'code': 500, 'created': '2020-03-05T15:38:04Z', 'message': 'No valid host was found. There are not enough hosts available.'}

Can you share if this happening a lot? if yes, can you also share nova and cinder logs?

Thanks

[Martin Kopec (mkopec)]

please provide more info as was requested in the comment #2 , but it looks like as an infra issue. I checked the last builds of grenade-py3 job and it seems it's working .. I'll mark this as Incomplete for now

[Martin Kopec (mkopec)]

the job is working fine now, I'm gonna close this. In case you'll experience the issue again, feel free to reopen this issue.