kolla-ansible does not work in a virtualenv

[Allen Gao]

It is not possible to use kolla-ansible command when kolla-ansible has been installed in a virtualenv.

Steps to reproduce:

$ virtualenv venv
$ source venv/bin/activate

# If installing from PyPI:
    $ pip install kolla-ansible
# Else if installing from a source checkout:
    $ pip install .

$ kolla-ansible prechecks

Expected result:

kolla-ansible executes the venv/share/kolla/ansible/site.yml playbook.

Actual result:

kolla-ansible uses an incorrect path to the site.yml playbook, venv/ansible/site.yml, causing ansible-playbook to fail.

Pre-deployment checking : ansible-playbook -i /home/mark/src/kolla-ansible/venv/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=precheck /home/mark/src/kolla-ansible/venv/ansible/site.yml
ERROR! the playbook: /home/mark/src/kolla-ansible/venv/ansible/site.yml could not be found
Command failed ansible-playbook -i /home/mark/src/kolla-ansible/venv/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=precheck /home/mark/src/kolla-ansible/venv/ansible/site.yml

Tested on:
- Ubuntu-16.04, kolla_ansible-4.0.0.0rc1-py2-none-any.whl (PyPI)
- CentOS 7.3, kolla-3.0.2 (source)

Out of scope:
This bug covers the use of a virtualenv on the Ansible control host, where the kolla-ansible package is installed. It does not cover the use of virtualenvs on the target hosts (which may or may not be the same host).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/262762

[OpenStack Infra (hudson-openstack)]

Change abandoned by Allen Gao (<email address hidden>) on branch: master
Review: https://review.openstack.org/262762

[Jay Chang (changsijay)]

I add another `elif` as below

```
    elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then
        BASEDIR="${VIRTUAL_ENV}/share/kolla"
```

It got correct BASEDIR, but failed on cannot find `docker` module as below :(

```
TASK [common : Creating log volume] ********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named docker
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_eKYY4o/ansible_module_kolla_docker.py\", line 195, in <module>\n import docker\nImportError: No module named docker\n", "module_stdout": "", "msg": "MODULE FAILURE"}
```

[Paul Bourke (pauldbourke)]

Have you ran 'pip install kolla' from within your virtualenv?

[Jay Chang (changsijay)]

looks fixed by below changes

```
$ diff /home/csj/.virtualenvs/kolla/bin/kolla-ansible /home/csj/.virtualenvs/temp/bin/kolla-ansible
12,13d11
< elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then
< BASEDIR="${VIRTUAL_ENV}/share/kolla"
73,77c71
< if [[ "${VIRTUAL_ENV}" != "" ]]; then
< EXTRA_OPTS="-e ansible_python_interpreter=${VIRTUAL_ENV}/bin/python"
< else
< EXTRA_OPTS=
< fi
---
> EXTRA_OPTS=
```

[Mark Goddard (mgoddard)]

The first part of the patch in comment #5 works for me:

$ diff /home/csj/.virtualenvs/kolla/bin/kolla-ansible /home/csj/.virtualenvs/temp/bin/kolla-ansible
12,13d11
< elif [[ ${dir_name} == "${VIRTUAL_ENV}/bin" ]]; then
< BASEDIR="${VIRTUAL_ENV}/share/kolla"

I don't think the latter part makes quite as much sense. We can't necessarily use the local VIRTUAL_ENV environment variable to determine the python interpreter to use on the remote hosts. If a virtualenv is in use on the remote hosts, this could be achieved by passing the -e argument to kolla-ansible manually.

[Mark Goddard (mgoddard)]

I read through the first review for this issue and took on board the point about the all-in-one needing to work. I think there's still a good reason to allow the control host to execute from within a virtualenv. When a Kolla package is unavailable or undesirable, it is advantageous to be able to install Kolla and its dependencies to a virtualenv on the control host to avoid touching the system python packages.

The use of a virtualenv on the target nodes is IMHO less important (although still possible without any code changes) as the required packages are commonly available in OS distros.

[Mark Goddard (mgoddard)]

Affects Kolla until (and including) Newton release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/434917

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/434917
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=dbc5967f7364489ba9a590969c3bba1a553b64fb
Submitter: Jenkins
Branch: master

commit dbc5967f7364489ba9a590969c3bba1a553b64fb
Author: Mark Goddard <email address hidden>
Date: Thu Feb 16 13:42:21 2017 +0000

    Support kolla-ansible installed in a virtualenv

    Currently it is not possible to use kolla-ansible when it has been
    installed in a virtualenv. Virtualenv-based installation may be
    desirable when a suitable system package (e.g. RPM, .deb) is not
    available, as it allows the user to install kolla-ansible and its
    dependencies without affecting the system-wide packages.

    This change checks for the presence of the $VIRTUAL_ENV environment
    variable to determine whether we are running in an virtualenv. Since
    kolla-ansible is not a python script, this comes with the caveat that we
    must source the virtualenv activation script ($VIRTUAL_ENV/bin/activate)
    before executing kolla-ansible.

    Change-Id: I19efb96605bee3b16df3a0b78cd681e5f15499f6
    Closes-Bug: #1530319

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 5.0.0.0b2 development milestone.