kolla-ansible-centos8-source-scenario-nfv Zuul test fails due to import error in tacker

[Yoshito Ito]

Tacker has changed its installation to use master branch of heat-translator [1] and tosca-parser [2] other than pip install with its requirements.txt. This change causes the missing modules in tacker-base docker image in kolla-ansible-centos8-source-scenario-nfv Zuul job.

To add some procedures to Dockerfile template for Tacker [3] to install the two required packages in the building of image.

[1] https://opendev.org/openstack/heat-translator
[2] https://opendev.org/openstack/tosca-parser
[3] https://opendev.org/openstack/kolla/src/branch/master/docker/tacker/tacker-base/Dockerfile.j2

The import error log:

2021-03-02T13:08:45.051944143Z + sudo -E kolla_set_configs
2021-03-02T13:08:45.110907072Z INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
2021-03-02T13:08:45.110932122Z INFO:__main__:Validating config file
2021-03-02T13:08:45.111027284Z INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
2021-03-02T13:08:45.111047815Z INFO:__main__:Copying service configuration files
2021-03-02T13:08:45.111209918Z INFO:__main__:Copying /var/lib/kolla/config_files/tacker.conf to /etc/tacker/tacker.conf
2021-03-02T13:08:45.111836143Z INFO:__main__:Setting permission for /etc/tacker/tacker.conf
2021-03-02T13:08:45.111933005Z INFO:__main__:Writing out command to execute
2021-03-02T13:08:45.112208691Z INFO:__main__:Setting permission for /var/log/kolla/tacker
2021-03-02T13:08:45.112384405Z INFO:__main__:Setting permission for /var/lib/tacker/vnfpackages
2021-03-02T13:08:45.112556599Z INFO:__main__:Setting permission for /var/lib/tacker/csar_files
2021-03-02T13:08:45.118161057Z ++ cat /run_command
2021-03-02T13:08:45.120809978Z + CMD='tacker-conductor --config-file /etc/tacker/tacker.conf'
2021-03-02T13:08:45.121014253Z + ARGS=
2021-03-02T13:08:45.121161846Z + sudo kolla_copy_cacerts
2021-03-02T13:08:45.134027610Z + [[ ! -n '' ]]
2021-03-02T13:08:45.134047661Z + . kolla_extend_start
2021-03-02T13:08:45.134267346Z ++ [[ ! -d /var/log/kolla/tacker ]]
2021-03-02T13:08:45.134681175Z +++ stat -c %a /var/log/kolla/tacker
2021-03-02T13:08:45.137287835Z ++ [[ 2755 != \7\5\5 ]]
2021-03-02T13:08:45.137299605Z ++ chmod 755 /var/log/kolla/tacker
2021-03-02T13:08:45.139117336Z ++ . /usr/local/bin/kolla_tacker_extend_start
2021-03-02T13:08:45.139345192Z + echo 'Running command: '\''tacker-conductor --config-file /etc/tacker/tacker.conf'\'''
2021-03-02T13:08:45.139428224Z Running command: 'tacker-conductor --config-file /etc/tacker/tacker.conf'
2021-03-02T13:08:45.139464454Z + exec tacker-conductor --config-file /etc/tacker/tacker.conf
2021-03-02T13:08:46.523346914Z Could not load 'glance.store.s3.Store': No module named 'boto3'
2021-03-02T13:08:46.531590252Z Could not load 's3': No module named 'boto3'
2021-03-02T13:08:46.802023554Z Traceback (most recent call last):
2021-03-02T13:08:46.802162487Z File "/var/lib/kolla/venv/bin/tacker-conductor", line 5, in <module>
2021-03-02T13:08:46.802175308Z from tacker.cmd.eventlet.conductor import main
2021-03-02T13:08:46.802181338Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/cmd/eventlet/conductor.py", line 13, in <module>
2021-03-02T13:08:46.802187698Z from tacker.conductor import conductor_server
2021-03-02T13:08:46.802192468Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/conductor/conductor_server.py", line 41, in <module>
2021-03-02T13:08:46.802197938Z from tacker import auth
2021-03-02T13:08:46.802222389Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/auth.py", line 21, in <module>
2021-03-02T13:08:46.802228159Z from tacker.api.vnflcm.v1 import router as vnflcm_router
2021-03-02T13:08:46.802232769Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/api/vnflcm/v1/router.py", line 20, in <module>
2021-03-02T13:08:46.802238069Z from tacker.api.vnflcm.v1 import controller as vnf_lcm_controller
2021-03-02T13:08:46.802242719Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/api/vnflcm/v1/controller.py", line 43, in <module>
2021-03-02T13:08:46.802247999Z from tacker.api.vnflcm.v1 import sync_resource
2021-03-02T13:08:46.802252550Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/api/vnflcm/v1/sync_resource.py", line 17, in <module>
2021-03-02T13:08:46.802257910Z from tacker.common import csar_utils
2021-03-02T13:08:46.802262400Z File "/var/lib/kolla/venv/lib/python3.6/site-packages/tacker/common/csar_utils.py", line 30, in <module>
2021-03-02T13:08:46.802267740Z from toscaparser.prereq.csar import CSAR
2021-03-02T13:08:46.802272340Z ModuleNotFoundError: No module named 'toscaparser'

[Mark Goddard (mgoddard)]

Tacker should not be using the master branch for these projects.

[Yoshito Ito (ito.yoshito)]

Thank you for the feedback.

Tacker team is now trying to find the best practice to use the latest commit of those packages. They are facing issues when they need to fix or extend functions in heat-translator or tosca-parser because Tacker highly depends on those projects and often finds that FT fails due to old packages of pip installation in the Zuul environment. This is the same situation when using kolla-ansible for Tacker installation. So I tried to change its installation process to uses master branch.

In fact, I'm not sure what to do other than using the master branch. It may be possible to publish newer release more frequently in heat-translator or tosca-parser, but I'm glad if I can hear your opinion.

[Mark Goddard (mgoddard)]

You might try using Zuul's dependency feature. You can then test a patch in your dependencies, but have the default behaviour of using a versioned release.