STABLE stream with wheels failed to docker images build

[JunyoungHwang]

Brief Description
-----------------
Hello.
I'm using StarlingX Master Branch.
Docker images build fails when building wheels using a stable stream.
When I build using the wheels that came out using build-wheel-tarball.sh,
the stx-glance image shows that there is no uWsgi version.
When I entered the running container when building the image, uWsgi 2.17.1 was not found in upper-requirements.txt.

In starlingx docs, it is said to build the wheel needed when STABLE build, but not match error occurs when building docker images.

Severity
--------
Major

Steps to Reproduce
------------------
$MY_REPO/build-tools/build-wheels/build-wheel-tarball.sh --os centos 7.5.1804 --stream stable

IP=$(ip addr | grep -w inet | grep -A 1 'eth0' | awk '{ print $2 }' | egrep -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}')
BUILD_WHEELS_STABLE_PY3="http://${IP}:8088//localdisk/loadbuild/${MYUNAME}/${PROJECT}/std/build-wheels-centos-stable/stx-centos-stable-wheels.tar"
BUILD_WHEELS_STABLE_PY2="http://${IP}:8088//localdisk/loadbuild/${MYUNAME}/${PROJECT}/std/build-wheels-centos-stable/stx-centos-stable-wheels-py2.tar"

$MY_REPO/build-tools/build-docker-images/build-stx-images.sh \
--os centos \
--stream stable \
--base junyounghwang/stx-centos:20210512-stable \
--wheels ${BUILD_WHEELS_STABLE_PY3} \
--wheels-py2 ${BUILD_WHEELS_STABLE_PY2} \

Expected Behavior
------------------
all wheels build, all docker images build

Actual Behavior
----------------
-rw-r--r-- 1 junyounghwang cgts 16530 May 13 02:38 api_proxy-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 76216 May 13 02:38 cgcs_patch-1.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 222095 May 13 02:38 cgtsclient-1.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 40924 May 13 02:38 controllerconfig-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 611337 May 13 02:38 distributedcloud-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 69822 May 13 02:38 distributedcloud_client-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 66605 May 13 02:38 fm-1.0.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 9424 May 13 02:38 fm_api-1.0.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 39947 May 13 02:38 fmclient-1.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 41909 May 13 02:38 fm_core-1.0-cp27-cp27mu-linux_x86_64.whl
-rw-r--r-- 1 junyounghwang cgts 21814 May 13 02:38 global-requirements.txt
-rw-r--r-- 1 junyounghwang cgts 433275 May 13 02:38 libvirt_python-4.7.0-cp27-cp27mu-linux_x86_64.whl
-rw-r--r-- 1 junyounghwang cgts 13844 May 13 02:38 nfv_client-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 3794 May 13 02:38 nfv_tools-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 661308 May 13 02:38 nfv_vim-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 3417 May 13 02:38 oidcauthtools-1.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 26909 May 13 02:38 python_cephclient-13.2.2.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 37529 May 13 02:38 sm_client-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 424596 May 13 02:38 starlingx_dashboard-1.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 1463876 May 13 02:38 sysinv-1.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 3729 May 13 02:38 tsconfig-1.0.0-py2.py3-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 33795 May 13 02:38 upper-constraints.txt
-rw-r--r-- 1 junyounghwang cgts 94374 May 13 02:38 windriver_nfv_common_plugins-1.0.0-py2-none-any.whl
-rw-r--r-- 1 junyounghwang cgts 97894 May 13 02:38 windriver_nfv_plugins-1.0.0-py2-none-any.whl

Exemple:(build-stx-images.sh result)
Could not find a version that satisfies the requirement uwsgi (from versions: none)
ERROR: No matching distribution found for uwsgi (from -c /tmp/wheels/upper-constraints.txt (line 21))
The command '/bin/sh -c /opt/loci/scripts/install.sh' returned a non-zero code: 1

Reproducibility
---------------
100% (And Same problem with the r/stx.4.0 branch.)

System Configuration
--------------------
N/A
Host OS: Ubuntu 18.04

Branch/Pull Time/Commit
-----------------------
root Master : commit ( a163d7723e659e89c37ec933d1b0f9aa638a6a73 )
tools Master : commit ( 7b5f3a45e663866a3c0ca3ca86eb3c92bc7f0210 )

Last Pass
---------
failing for at least Now(05.13.2021) and 03.17.2021

Timestamp/Logs
--------------
BUILD_INFO
OS="centos"
JOB="n/a"
BUILD_BY="junyounghwang"
BUILD_NUMBER="n/a"
BUILD_HOST="83269b08d1c0"
BUILD_DATE="2021-05-12 02:38:08 +0000"

Ex) stx-glance image build log
==> Include an attachment

Test Activity
-------------
Build

Workaround
----------

[JunyoungHwang (juuns)]

Running the docker-build-wheel.sh script in stable builds whl, but it seems to be a problem because the build result is not displayed in the base directory.

[JunyoungHwang (juuns)]

I tried the following.

<Root Project>
1. build-tools/build-wheels/docker/centos-dockerfile
ADD====================>
yum install libev libev-devel
Why: cassandra.io.libevwrapper compile failed when python2 wheels build
Result Succeeded

2. build-tools/build-wheels/docker/centos-dockerfile
ADD====================>
RUN mkdir /wheels
Why: docker run ${RUN_ARGS[@]} -v ${BUILD_OUTPUT_PATH}:/wheels ${BUILD_IMAGE_NAME} /bin/bash /docker-build-wheel.sh in build-wheels/build-base-wheels.sh /wheels Directory not created. Also, whl is not included (created) in base and base-py2 of ${BUILD_OUTPUT_PATH}.

3. build-tools/build-wheels/build-base-wheels.sh
CHANGE=================>
181: #rm -f $f
why: Comment out the entry on line 181 that deletes the whl of base and base-py2 as a temporary method.
Because there is a problem that wheels made in docker-build-wheel.sh are not saved in base and base-py2, I copied them manually. Therefore, I added a comment because if I delete it, it will not build properly.
The manual way to do this is to rebuild the wheels after copying to "Container ID running scp docker-build-wheel.sh:/wheels ${BUILD_OUTPUT_PATH} or ${BUILD_OUTPUT_PATH_PY2}.
Result: Wheels Build success

Currently, we are testing the above items by cherry-picking the modified repo from the master branch to the r/stx.4.0 branch of the tools project. (Stable)
However, the same problem as above is seen in the master branch (tools, root), so it seems that it needs to be fixed.

Currently: docker image build works normally in the r/stx.4.0 branch.

Please note: Please modify it so that compatibility and smooth build of multiple systems can be achieved.

[JunyoungHwang (juuns)]

Commit this https://github.com/starlingx/root/commit/2bdac9ac3b09332604919c8e7d39c1676c106ae5#diff-fc9faf10d0ec4475b11e869371572672b020a3bffbf8a52d65e3a8dc0404c564
  Has a problem. The docker run -v command on lines 332 and 338 creates the wheel built in the BUILD_OUTPUT_PATH of the Host. Therefore, there is a problem that the built wheel cannot be referenced in the container executed with tb.sh.

[JunyoungHwang (juuns)]

It seems that you should use the --volumes-from option rather than the docker run -v option.

[JunyoungHwang (juuns)]

I have modified it as follows.

329 # setting docker container
330 BUILD_CONTAINER=`docker ps -a -n 1 --filter ancestor="builder-ecobox-wheelbuilder:centos-stable" --format '{{.ID}}'`
331
332 notice "building python3 wheels"
333 log_prefix "[python3] " \
334 with_retries ${MAX_ATTEMPTS} \
335 docker run ${RUN_ARGS[@]} -v ${BUILD_OUTPUT_PATH}:/wheels ${BUILD_IMAGE_NAME} /docker-build-wheel.sh
336 BUILD_STATUS=$?
337 docker start ${BUILD_CONTAINER} && docker cp ${BUILD_CONTAINER}:/wheels/. ${BUILD_OUTPUT_PATH}
338
339 notice "building python2 wheels"
340 log_prefix "[python2] " \
341 with_retries ${MAX_ATTEMPTS} \
342 docker run ${RUN_ARGS[@]} -v ${BUILD_OUTPUT_PATH_PY2}:/wheels --env PYTHON=python2 ${BUILD_IMAGE_NAME} /docker-build-wheel.s h
343 BUILD_STATUS_PY2=$?
344 docker start ${BUILD_CONTAINER} && docker cp ${BUILD_CONTAINER}:/wheels/. ${BUILD_OUTPUT_PATH_PY2}

The wheel now builds properly.

[JunyoungHwang (juuns)]

!!!!!!!!!!!!!!!!!!
There seems to be a problem with the "build-base-wheels.sh" script.
When using docker run -v, it seems to mount from the host's path. The host's /localdisk path is created.

Therefore, it has been modified as follows.
In the "build-base-wheels.sh" script,
  docker run ${RUN_ARGS[@]} -v /home/$USER/starlingx/workspace${BUILD_OUTPUT_PATH}:/wheels ${BUILD_IMAGE_NAME} /docker-build-wheel.sh

docker run ${RUN_ARGS[@]} -v /home/$USER/starlingx/workspace${BUILD_OUTPUT_PATH_PY2}:/wheels --env PYTHON=python2 ${BUILD_IMAGE_NAME} /docker-build-wheel.sh

After this modification, the whl file was not normally created in the host's /localdisk, but was created normally in the normal mount path, $MY_BUILD_DIR/std in the container.

[Ghada Khalil (gkhalil)]

screening: marking as low priority due to lack of activity