Tempest Container Image is not building successfully using buildah

[yatin]

buildah based container-build job is running successfully[1] even if tempest image is not building successfully .

More details can be found in the debug patch[2]

When reproduced locally using below steps i got below ERROR:-
error building at step {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin KOLLA_BASE_DISTRO=centos KOLLA_INSTALL_TYPE=binary KOLLA_INSTALL_METATYPE=rdo KOLLA_DISTRO_PYTHON_VERSION=2.7 PS1=$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ] Command:workdir Args:[/tempest] Flags:[] Attrs:map[] Message:WORKDIR /tempest Original:WORKDIR /tempest}: error ensuring container path "/tempest": mkdir /var/lib/containers/storage/overlay/0c896ea578d5d9018fee16a0ee19573d3dda4a8694f22b810a3f3f3b0c28e0ac/merged/tempest: file exists
Steps ran locally:-
mkdir tempest
pushd tempest
curl -O http://logs.openstack.org/08/652508/5/check/tripleo-build-containers-centos-7-buildah/176aba8/logs/buildah-builds/kolla-UdjL2R/docker/tempest/Dockerfile.gz
curl -O http://logs.openstack.org/08/652508/5/check/tripleo-build-containers-centos-7-buildah/176aba8/logs/buildah-builds/kolla-UdjL2R/docker/tempest/extend_start.sh.txt.gz
gunzip Dockerfile.gz
mv extend_start.sh.txt.gz extend_start.sh
sed -i 's/127.0.0.1:8787/docker.io/' Dockerfile
popd
sudo buildah bud --tls-verify=False --logfile tempest/tempest-build.log -t 127.0.0.1:8787/tripleomaster/centos-binary-tempest:be90d93c3c5f77f428d12a9a8a2ef97b9dada8f3_5b53d5ba tempest

Docker build job successfully builts tempest image, so issue seems specific to buildah(when running WORKDIR /tempest when /tempest already exists because of previous RUN statement: ln -sf /usr/share/openstack-tempest /tempest).

[1] http://zuul.openstack.org/builds?job_name=tripleo-build-containers-centos-7-buildah
[2] https://review.openstack.org/#/c/652508/

[yatin (yatinkarel)]

The issue is already reported in buildah:- https://github.com/containers/buildah/issues/1439

[Emilien Macchi (emilienm)]

The fix is likely going to be https://github.com/containers/buildah/pull/1443

[yatin (yatinkarel)]

The issue is fixed in buildah master with PR:- https://github.com/containers/buildah/pull/1558, now we need a new release and build for buildah.

[wes hayutin (weshayutin)]

can be moved to fix released?

[yatin (yatinkarel)]

Yes can be closed, as buildah-1.8.1 with the fix is in master and stein where buildah jobs are running.

[yatin (yatinkarel)]

buildah-1.8.1 which contain the fix is in both stein and master delorean deps repo. Closing the bug.

[yatin (yatinkarel)]

For record with https://github.com/containers/buildah/commit/c0633e38da4134135d35d32b56194790ca8acecd it was completely fixed for both dangling and non dangling symlink. So buildah-1.8.3 is needed.