bootstrap-servers with ven: Could not import python modules: apt, apt_pkg

[Adrian Andreias]

I'm using kolla and kolla-ansible from git kolla-ansible (commit 1a7478d2f origin/stable/victoria)

Running

kolla-ansible -i ./all-in-one bootstrap-servers

on Ubuntu 20.04 with ansible_python_interpreter: "/path/env/bin/python" in /etc/kolla/globals.yml
fails with error:

TASK [baremetal : Install apt packages] ********************************************************************************************************************************************************************
[WARNING]: Updating cache and auto-installing missing dependency: python3-apt
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Could not import python modules: apt, apt_pkg. Please install python3-apt package."}

This fixed after recreating the virtual environment option --system-site-packages:

python3 -m venv env --system-site-packages

Should this option be added to docs here https://docs.openstack.org/kolla-ansible/victoria/user/virtual-environments.html ?

[Michal Nasiadka (mnasiadka)]

Are you bootstrapping localhost in that run?
Probably python3-apt needs to be installed on the destination servers - it would be good to understand that before we update the docs/code.

[Radosław Piliszek (yoctozepto)]

Following up on the above note by Michał - you mean you are using virtualenv on the *target* hosts, right?

[Adrian Andreias (adrian-fleio)]

Yes, localhost to localhost.
Yes, virtualenv on target host. That's what this in globals.yml is for, right?

ansible_python_interpreter: "/path/env/bin/python"

[Radosław Piliszek (yoctozepto)]

Yes, sure thing. I guess you are right about the command to generate venv then. As otherwise Ansible modules are grumpy.

[Adrian Andreias (adrian-fleio)]

I've tried to reproduce this step by step in a fresh Ubuntu 20.04 install, paying more attention to the docs.

And I've noticed this issue is actually mentioned in https://docs.openstack.org/kolla-ansible/victoria/user/virtual-environments.html#target-hosts

> Kolla-ansible provides support for creating a python virtual environment on the target hosts as part of the bootstrap-servers command. The path to the virtualenv is configured via the virtualenv variable, and access to site-packages is controlled via virtualenv_site_packages. Typically we will need to enable use of system site-packages from within this virtualenv, to support the use of modules such as yum, apt, and selinux, which are not available on PyPI.

Tried to add these to /etc/kolla/globals.yml :

virtualenv_site_packages: yes
virtualenv: /root/targethost/env
ansible_python_interpreter: /root/targethost/env/bin/python

but didn't help, since on "kolla-ansible -i ./all-in-one bootstrap-servers" I get:

TASK [Gathering Facts] *************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"setup": {"failed": true, "module_stderr": "/bin/sh: 1: /root/targethost/env/bin/python: not found\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}}, "msg": "The following modules failed to execute: setup\n"}

In which file should I add those? (I'm am not that familiar with Ansible either).

I can get over these issues as mentioned above. But I'd add more details to kolla-ansible docs to have a smooth install on Ubuntu 20.04, it might help someone in the future.

Some other minor issues I found during install are:

# this is needed to create venv, but not mentioned in docs:
apt-get install python3-venv

# this was required, does not appear in docs. Useful for mindless copy/pasting :)
mkdir /etc/ansible

[Adrian Andreias (adrian-fleio)]

From my understanding of the docs, kolla-asible should create the virtual environment:

> Kolla-ansible provides support for creating a python virtual environment on the target hosts as part of the bootstrap-servers command. The path to the virtualenv is configured via the virtualenv variable, and access to site-packages is controlled via virtualenv_site_packages.

https://docs.openstack.org/kolla-ansible/latest/user/virtual-environments.html#target-hosts

I've added this to the top of /etc/kolla/globals.yml:

virtualenv_site_packages: yes
virtualenv: /root/targethost/env
ansible_python_interpreter: /root/targethost/env/bin/python

And I get:

# kolla-ansible -i ./all-in-one bootstrap-serverss
Bootstrapping servers : ansible-playbook -i ./all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=bootstrap-servers /root/kolla/env/share/kolla-ansible/ansible/kolla-host.yml
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [Gather facts for all hosts] **************************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"setup": {"failed": true, "module_stderr": "/bin/sh: 1: /root/targethost/env/bin/python: not found\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}}, "msg": "The following modules failed to execute: setup\n"}

PLAY RECAP *************************************************************************************************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

Command failed ansible-playbook -i ./all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=bootstrap-servers /root/kolla/env/share/kolla-ansible/ansible/kolla-host.yml

I can update the docs and even try to fix the code. But I'd need to know if I'm doing something wrong or this is a bug.

[Adrian Andreias (adrian-fleio)]

Actually this works if bootstrap-servers is ran without ansible_python_interpreter:

virtualenv_site_packages: yes
virtualenv: /root/targethost/env
#ansible_python_interpreter: /root/targethost/env/bin/python

This creates env and the subsequent commands can be run with ansible_python_interpreter.

And this is actually specified in docs:

When executing kolla-ansible commands other than bootstrap-servers, the variable ansible_python_interpreter should be set to the python interpreter installed in virtualenv.

[Mark Goddard (mgoddard)]

I think this happens if you create a virtualenv on localhost to install kolla-ansible in, then use the same virtual environment for localhost as a target.

[Adrian Andreias (adrian-fleio)]

I wasn't using the same venv, but /root/kolla-ansible/venv for kolla-ansible and /root/targethost/env for Ansible on "target".

Anyhow, paying attention to docs avoids the problems:

> Typically we will need to enable use of system site-packages from within this virtualenv, to support the use of modules such as yum, apt, and selinux, which are not available on PyPI.

> When executing kolla-ansible commands other than bootstrap-servers, the variable ansible_python_interpreter should be set to the python interpreter installed in virtualenv.

I was also mixing the `python3 -m venv /path/to/venv` example command for control host with creating venv on target (the latter is actually done by Ansible, not manually).

I think the issue may be closed.

Thanks