Ensure dependency packages installed on target hosts/deployed-servers for tripleo-ansible modules to function

[Bogdan Dobrelya]

Since most of tripleo-ansible modules do 'import foobar', we should ensure/validate that we have the correct python packages installed on target hosts. Some of them, like python3-dmidecode may be in base Centos8 images. But some may not, especially for custom deployed-servers provided by users for deployments.

Those packages must be tracked and ensured to be installed by tripleo (preferred), or validated deploy-time (nah...), or at least documented as the modules get created or changed by devs.

That also applies to adding action plugins' deps for python-tripleoclient or tripleo-ansible perhaps

[Bogdan Dobrelya (bogdando)]

Note that we also have to manage test_deps, like that https://review.opendev.org/#/c/749825. Unifiying the approach for testing and deployment dependencies installation would be nice.

[Cédric Jeanneret (cjeanner)]

Hello,

We can of course validate the presence of packages with a simple validation in tripleo-validations repository.

We can either:
- create a new validation
- or use the existing "package_version"

The latter has my preference, since it allows to check if selected packages are installed, and if their version matches the requirements.
Now, it's more a matter of getting the actual list... ?

[Bogdan Dobrelya (bogdando)]

10:05:40 AM - cjeanner: bogdando: maybe deployed-server/deployed-server.yaml could import some new role from tripleo-ansible.
10:06:15 AM - bogdando: cjeanner: yeah, and prolly the same role can be reused when managing test deps

[Bogdan Dobrelya (bogdando)]

as noted in the description, the actual list is manual work for devs to maintain as they develop modules

[Bogdan Dobrelya (bogdando)]

And we should get some meta-package in RDO, and just say "install overcloud-tripleo-deps.rpm" there. While all the needful (including podman et al pinnings) will be handled as that package dependenies.

[Alex Schultz (alex-schultz)]

Since we need to support both overcloud image deployed and pre-deployed servers. We end up having to track these in two places. Missing deps need to go in the image and the tripleo_bootstrap role. A meta package would be nice but that's a bunch of overhead.

[Bogdan Dobrelya (bogdando)]

Here is a list of missing python libs if the used tripleo-ansible imports ran in a scratch centos-8 container:

$ grep -irPh '^(import|from)' ./tripleo_ansible/ansible_plugins/modules | grep -Pv '[tm] (tripleo_|ansible\.)' | sort -u > list

$ cat list | xargs -n1 -I {} python3 -c '{}' 2>&1 | grep '^Mo' | sort -u
ModuleNotFoundError: No module named 'future'
ModuleNotFoundError: No module named 'heatclient'
ModuleNotFoundError: No module named 'metalsmith'
ModuleNotFoundError: No module named 'openstack'
ModuleNotFoundError: No module named 'yaml'

[Alex Schultz (alex-schultz)]

These are needed on the undercloud and not the overcloud (except for yaml), but yes they should be added to the tripleo-ansible rpm if not currently defined somewhere

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/758818

[Bogdan Dobrelya (bogdando)]

yeah, so only pyyaml remains in the list... addressed that

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/758818
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=47612998c756cc589b74f763b2227303caa39222
Submitter: Zuul
Branch: master

commit 47612998c756cc589b74f763b2227303caa39222
Author: Bogdan Dobrelya <email address hidden>
Date: Mon Oct 19 14:57:18 2020 +0200

    Ensure yaml pkg imported in modules is installed

    Ansible modules rely on custom imports such as yaml. Since we need to
    support both overcloud image deployed and pre-deployed servers, we end
    up having to track these in two places. Missing deps need to go in the
    image and the tripleo_bootstrap role. This change addressed the latter.

    Closes-bug: #1894957
    Change-Id: Ied27b97db9aab16dd92aa9dc99e93d4834d3c633
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-ansible 3.0.0 release.