kolla-ansible does not work in a virtualenv

Bug #1530319 reported by Allen Gao on 2015-12-31
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Mark Goddard

Bug Description

It is not possible to use kolla-ansible command when kolla-ansible has been installed in a virtualenv.

Steps to reproduce:

$ virtualenv venv
$ source venv/bin/activate

# If installing from PyPI:
    $ pip install kolla-ansible
# Else if installing from a source checkout:
    $ pip install .

$ kolla-ansible prechecks

Expected result:

kolla-ansible executes the venv/share/kolla/ansible/site.yml playbook.

Actual result:

kolla-ansible uses an incorrect path to the site.yml playbook, venv/ansible/site.yml, causing ansible-playbook to fail.

Pre-deployment checking : ansible-playbook -i /home/mark/src/kolla-ansible/venv/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=precheck /home/mark/src/kolla-ansible/venv/ansible/site.yml
ERROR! the playbook: /home/mark/src/kolla-ansible/venv/ansible/site.yml could not be found
Command failed ansible-playbook -i /home/mark/src/kolla-ansible/venv/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=precheck /home/mark/src/kolla-ansible/venv/ansible/site.yml

Tested on:
- Ubuntu-16.04, kolla_ansible- (PyPI)
- CentOS 7.3, kolla-3.0.2 (source)

Out of scope:
This bug covers the use of a virtualenv on the Ansible control host, where the kolla-ansible package is installed. It does not cover the use of virtualenvs on the target hosts (which may or may not be the same host).

Allen Gao (wanlong-gao) on 2015-12-31
Changed in kolla:
assignee: nobody → Wanlong Gao (wanlong-gao)

Fix proposed to branch: master
Review: https://review.openstack.org/262762

Changed in kolla:
status: New → In Progress

Change abandoned by Allen Gao (<email address hidden>) on branch: master
Review: https://review.openstack.org/262762

Sam Yaple (s8m) on 2016-03-21
Changed in kolla:
status: In Progress → Opinion

I add another `elif` as below

    elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then

It got correct BASEDIR, but failed on cannot find `docker` module as below :(

TASK [common : Creating log volume] ********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named docker
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_eKYY4o/ansible_module_kolla_docker.py\", line 195, in <module>\n import docker\nImportError: No module named docker\n", "module_stdout": "", "msg": "MODULE FAILURE"}

Paul Bourke (pauldbourke) wrote :

Have you ran 'pip install kolla' from within your virtualenv?

Changed in kolla:
status: Opinion → Incomplete
CSJ (changsijay) wrote :

looks fixed by below changes

$ diff /home/csj/.virtualenvs/kolla/bin/kolla-ansible /home/csj/.virtualenvs/temp/bin/kolla-ansible
< elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then
< BASEDIR="${VIRTUAL_ENV}/share/kolla"
< if [[ "${VIRTUAL_ENV}" != "" ]]; then
< EXTRA_OPTS="-e ansible_python_interpreter=${VIRTUAL_ENV}/bin/python"
< else
< fi

Changed in kolla:
importance: Undecided → Wishlist
Mark Goddard (mgoddard) wrote :

The first part of the patch in comment #5 works for me:

$ diff /home/csj/.virtualenvs/kolla/bin/kolla-ansible /home/csj/.virtualenvs/temp/bin/kolla-ansible
< elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then
< BASEDIR="${VIRTUAL_ENV}/share/kolla"

I don't think the latter part makes quite as much sense. We can't necessarily use the local VIRTUAL_ENV environment variable to determine the python interpreter to use on the remote hosts. If a virtualenv is in use on the remote hosts, this could be achieved by passing the -e argument to kolla-ansible manually.

Mark Goddard (mgoddard) wrote :

I read through the first review for this issue and took on board the point about the all-in-one needing to work. I think there's still a good reason to allow the control host to execute from within a virtualenv. When a Kolla package is unavailable or undesirable, it is advantageous to be able to install Kolla and its dependencies to a virtualenv on the control host to avoid touching the system python packages.

The use of a virtualenv on the target nodes is IMHO less important (although still possible without any code changes) as the required packages are commonly available in OS distros.

Mark Goddard (mgoddard) wrote :

Affects Kolla until (and including) Newton release.

Changed in kolla:
assignee: Allen Gao (wanlong-gao) → Mark Goddard (mgoddard)
affects: kolla → kolla-ansible
Changed in kolla:
status: New → Incomplete
Mark Goddard (mgoddard) on 2017-02-16
summary: - kolla ansible can not work in virtualenv
+ kolla-ansible does not work in a virtualenv
Mark Goddard (mgoddard) on 2017-02-16
description: updated
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/434917

Changed in kolla-ansible:
status: Incomplete → In Progress

Reviewed: https://review.openstack.org/434917
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=dbc5967f7364489ba9a590969c3bba1a553b64fb
Submitter: Jenkins
Branch: master

commit dbc5967f7364489ba9a590969c3bba1a553b64fb
Author: Mark Goddard <email address hidden>
Date: Thu Feb 16 13:42:21 2017 +0000

    Support kolla-ansible installed in a virtualenv

    Currently it is not possible to use kolla-ansible when it has been
    installed in a virtualenv. Virtualenv-based installation may be
    desirable when a suitable system package (e.g. RPM, .deb) is not
    available, as it allows the user to install kolla-ansible and its
    dependencies without affecting the system-wide packages.

    This change checks for the presence of the $VIRTUAL_ENV environment
    variable to determine whether we are running in an virtualenv. Since
    kolla-ansible is not a python script, this comes with the caveat that we
    must source the virtualenv activation script ($VIRTUAL_ENV/bin/activate)
    before executing kolla-ansible.

    Change-Id: I19efb96605bee3b16df3a0b78cd681e5f15499f6
    Closes-Bug: #1530319

Changed in kolla-ansible:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers