As can be seen in https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable1-cs8
The jobs periodic-tripleo-ci-build-containers-ubi-8-push-wallaby and periodic-tripleo-ci-build-containers-ubi-8-quay-push-wallaby are failing because of missing containers such as nova-api, nova-scheduler among others.
It is hard right now to find the root cause because there is no logs for these builds, only at the end of the build containers log, you can notice this:
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder [-] [Errno 2] No such file or directory: '/home/zuul/container-builds/9e4b8ca8-1450-485b-8eb7-dfa2b42b0c35/base/os/swift-base/swift-proxy-server/swift-proxy-server-build.log': FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/container-builds/9e4b8ca8-1450-485b-8eb7-dfa2b42b0c35/base/os/swift-base/swift-proxy-server/swift-proxy-server-build.log'
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder Traceback (most recent call last):
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tripleo_common/image/builder/buildah.py", line 143, in _generate_container
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder self._find_container_dir(container_name))
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tenacity/__init__.py", line 333, in wrapped_f
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder return self(f, *args, **kw)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tenacity/__init__.py", line 423, in __call__
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder do = self.iter(retry_state=retry_state)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tenacity/__init__.py", line 372, in iter
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder raise retry_exc.reraise()
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tenacity/__init__.py", line 189, in reraise
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder raise self.last_attempt.result()
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder return self.__get_result()
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder raise self._exception
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tenacity/__init__.py", line 426, in __call__
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder result = fn(*args, **kwargs)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/home/zuul/workspace/venv_build/lib64/python3.6/site-packages/tripleo_common/image/builder/buildah.py", line 175, in build
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder pathlib.Path(logfile).touch()
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/usr/lib64/python3.6/pathlib.py", line 1238, in touch
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder fd = self._raw_open(flags, mode)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/usr/lib64/python3.6/pathlib.py", line 1046, in _raw_open
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder return self._accessor.open(self, flags, mode)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder File "/usr/lib64/python3.6/pathlib.py", line 387, in wrapped
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder return strfunc(str(pathobj), *args)
2022-09-05 14:09:31 | 2022-09-05 14:09:31.610 39724 ERROR tripleo_common.image.builder.buildah.BuildahBuilder FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/container-builds/9e4b8ca8-1450-485b-8eb7-dfa2b42b0c35/base/os/swift-base/swift-proxy-server
However, since we can't find the logs for these builds, it's hard to identify the reason it is failing.
I tried to run the job in a testproject and hold the node to debug more, but the job passes, however, on the pipeline it is always failing with timeout.