Broken source builds after plugin introduction in openstack-base image

[Tadas Sutkaitis]

After CADF plugin introduction in 18.0.0 release:

"ADD plugins-archive /openstack-base-source"

Building from git sources is broken because source path under /openstack-base-source for openstack/requests is named differently

then /openstack/requests* and has name used with naming convention used for source builds -> /openstack/openstack-base-archive-XXXX

Next step in Dockerfile is broken because of this:

"RUN ln -s openstack-base-source/*requirements* /requirements" and upper-constrains.txt is not copied.

[Tadas Sutkaitis (t42x)]

Proposed fix https://review.opendev.org/c/openstack/kolla/+/922077

[Sven Kieske (s-kieske)]

Hi and thanks for the patch!

Can you please report what the actual error is you are seeing? I'm currently trying to replicate this on master branch, but I get unrelated other errors about distutils module missing from pbr.

[Tadas Sutkaitis (t42x)]

ERROR:kolla.common.utils.openstack-base:Step 5/11 : ADD openstack-base-archive /openstack-base-source
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> b1fe34250c70
ERROR:kolla.common.utils.openstack-base:Step 6/11 : ADD plugins-archive /openstack-base-source
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> 9b25743f0493
ERROR:kolla.common.utils.openstack-base:Step 7/11 : RUN ln -s openstack-base-source/plugins/* /pycadf && mkdir -p /etc/pycadf && cp /pycadf/etc/pycadf/* /etc/pycadf/
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Running in 1795afb3e756
ERROR:kolla.common.utils.openstack-base:---> Removed intermediate container 1795afb3e756
ERROR:kolla.common.utils.openstack-base:---> fa38c0449106
ERROR:kolla.common.utils.openstack-base:Step 8/11 : RUN ln -s openstack-base-source/*requirements* /requirements && sed -e "/^ovs===/d" -i requirements/upper-constraints.txt && mkdir -p /var/lib/kolla && python3 -m venv --system-site-packages /var/lib/kolla/venv
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Running in 7968db7ce998
ERROR:kolla.common.utils.openstack-base:sed: can't read requirements/upper-constraints.txt: No such file or directory

ERROR:kolla.common.utils.openstack-base:---> Removed intermediate container 7968db7ce998
ERROR:kolla.common.utils.openstack-base:Unknown container engine error when building

And if I add extra step in Dockerfile to list content:

RUN ls -al /openstack-base-source

In output I see:

ERROR:kolla.common.utils.openstack-base:Step 7/12 : RUN ls -al /openstack-base-source
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Running in 7a03c5fc1618
ERROR:kolla.common.utils.openstack-base:total 16
drwxr-xr-x 1 root root 4096 Jul 16 10:28 .
drwxr-xr-x 1 root root 4096 Jul 16 10:28 ..
drwxr-xr-x 9 root root 4096 Jul 16 10:28 openstack-base-archive-d9e1ea888b9975dc0cfb7ebd1d5cf0ad65331b34
drwxr-xr-x 3 root root 4096 Jul 16 10:28 plugins

[Sven Kieske (s-kieske)]

I'm not sure the error you are seeing is related to this patch you mention:

https://review.opendev.org/c/openstack/kolla/+/904576

I can't reproduce this locally with kolla-build from current master branch building openstack-base, it builds fine.

Can you maybe share your kolla-build.conf and which distributions or with with container engine (versions) you are building?

Thank you!

[Tadas Sutkaitis (t42x)]

To reproduce this bug please set "openstack-base" source type to git like this:

[openstack-base]
location = https://github.com/openstack/requirements.git
reference = 94f3181ff91c6f7a5bb18a202d19afddd0565358 # (last commit from master or stable/2024.1)
type = git

In this case path for "requirements" under /openstack-base-source will be "/openstack-base-source/openstack-base-archive-d9e1ea888b9975dc0cfb7ebd1d5cf0ad65331b34" and not "/openstack-base-source/requirements-xxx". This is why step in Dockerfile "RUN ln -s openstack-base-source/*requirements* /requirements \" silently copies nothing.

[Tadas Sutkaitis (t42x)]

I searched the code and used same behavior like in every other Dockerfile in Kolla project. I mean we always do "ADD plugins-archive /" + additional logic, but never using same path for "plugin-archive" and "project-x-source"

[Sven Kieske (s-kieske)]

I can confirm that I'm now also seeing a local error when building openstack-base.

However my error at least seems to slightly differ from yours, or did you maybe not post the complete error message?

Thanks for providing reproducer steps!

[Tadas Sutkaitis (t42x)]

I'm using docker engine:

INFO:kolla.common.utils:Attempt number: 4 to run task: BuildTask(openstack-base)
INFO:kolla.common.utils.openstack-base:Building started at 2024-07-18 12:32:42.689756
INFO:kolla.common.utils.openstack-base:Clone dir /var/folders/cm/y_rphw8s2gd7rm0z9fb8cd0c0000gn/T/kolla-2024-07-18_12-30-51_lajr6cs0/docker/openstack-base/openstack-base-archive-94f3181ff91c6f7a5bb18a202d19afddd0565358 exists. Removing it.
INFO:kolla.common.utils.openstack-base:Directory /var/folders/cm/y_rphw8s2gd7rm0z9fb8cd0c0000gn/T/kolla-2024-07-18_12-30-51_lajr6cs0/docker/openstack-base/additions already exist. Skipping.
ERROR:kolla.common.utils.openstack-base:[WARNING]: Empty continuation line found in: RUN ln -s openstack-base-source/*requirements* /requirements && sed -e "/^ovs===/d" -i requirements/upper-constraints.txt && mkdir -p /var/lib/kolla && python3 -m venv --system-site-packages /var/lib/kolla/venv

ERROR:kolla.common.utils.openstack-base:[WARNING]: Empty continuation lines will become errors in a future release.

ERROR:kolla.common.utils.openstack-base:Step 1/10 : FROM quay.io/xxxxxxxx/kolla/base:2024.1
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> eff75fe2e854
ERROR:kolla.common.utils.openstack-base:Step 2/10 : LABEL maintainer="Tadas <email address hidden>" name="openstack-base" build-date="20240718"
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Using cache
ERROR:kolla.common.utils.openstack-base:---> 1a61d3793886
ERROR:kolla.common.utils.openstack-base:Step 3/10 : RUN dnf config-manager --enable crb || true
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Using cache
ERROR:kolla.common.utils.openstack-base:---> 8fd4e1ea96bc
ERROR:kolla.common.utils.openstack-base:Step 4/10 : RUN dnf -y install gcc gcc-c++ git httpd iproute libffi-devel libxml2-devel libxslt-devel libyaml-devel make mod_ssl openssl openssl-devel pcre-devel postgresql postgresql-devel python3-devel python3-mod_wsgi sqlite-devel zip && dnf clean all && rm -rf /var/cache/dnf
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Using cache
ERROR:kolla.common.utils.openstack-base:---> 90e1caadb291
ERROR:kolla.common.utils.openstack-base:Step 5/10 : ADD plugins-archive /openstack-base-source
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Using cache
ERROR:kolla.common.utils.openstack-base:---> 720a7bb530fd
ERROR:kolla.common.utils.openstack-base:Step 6/10 : RUN ln -s openstack-base-source/plugins/* /pycadf && mkdir -p /etc/pycadf && cp /pycadf/etc/pycadf/* /etc/pycadf/
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.openstack-base:---> Using cache
ERROR:kolla.common.utils.openstack-base:---> 89865a2bda9a
ERROR:kolla.common.utils.openstack-base:Step 7/10 : RUN ln -s openstack-base-source/*requirements* /requirements && sed -e "/^ovs===/d" -i requirements/upper-constraints.txt && mkdir -p /var/lib/kolla && python3 -m venv --system-site-packages /var/lib/kolla/venv
ERROR:kolla.common.utils.openstack-base:
ERROR:kolla.common.utils.opensta...