kolla-ansible does not work with pyenv-virtualenv

Bug #1903887 reported by zengchen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
Medium
Radosław Piliszek

Bug Description

(kolla) [kolla@deployhost]$ kolla-ansible -i nodes deploy -v
Deploying Playbooks : ansible-playbook -i nodes -e /etc/kolla/globals.yml -e /etc/kolla/passwords.yml -e kolla_action=deply /home/kolla/.pyenv/versions/kolla/ansible/site.yml --verbose
Using /home/kolla/.ansible.cfg as config file
ERROR! the playbook: /home/kolla/.pyenv/versions/kolla/ansible/site.yml could not be found
Command failed ansible-playbook -i nodes -e /etc/kolla/globals.yml -e /etc/kolla/passwords.yml -e kolla_action=deply /home/kolla/.pyenv/versions/kolla/ansible/site.yml --verbose

(kolla) [kolla@deployhost]$ pyenv versions
  system
  3.7.9
  3.7.9/envs/kolla
* kolla (set by /home/kolla/.python-versions

(kolla) [kolla@deployhost]$ ll /home/kolla/.pyenv/versions/kolla
lrwxrwxrwx. 1 kolla kolla 44 Nov 14 2019 /home/kolla/.pyenv/versions/kolla -> /home/kolla/.pyenv/versions/3.7.9/envs/kolla

then VIRTUAL_ENV env is /home/kolla/.pyenv/versions/kolla, which is a link of /home/kolla/.pyenv/versions/3.7.9/envs/kolla, so kolla-ansible check virtualenv failed and give a wrong dir prefix.

os version: centos 7.8.2003
python version: 3.7.9
python virtualenv: created by pyenv
kolla-ansible version: pip kolla-ansible==9.2.0

Revision history for this message
zengchen (zengchen228) wrote :
Changed in kolla-ansible:
assignee: nobody → zengchen (zengchen228)
assignee: zengchen (zengchen228) → nobody
assignee: nobody → zengchen (zengchen228)
zengchen (zengchen228)
description: updated
zengchen (zengchen228)
description: updated
Changed in kolla-ansible:
status: New → In Progress
Revision history for this message
Radosław Piliszek (yoctozepto) wrote :

This is pyenv-virtualenv, no? And not just pyenv.

Revision history for this message
zengchen (zengchen228) wrote :

set the workdir use the command: pyenv local kolla, it will auto set pyenv virtual env to the alias.

Revision history for this message
Radosław Piliszek (yoctozepto) wrote :

Ok, I debugged this and it seems it's a quirk with one of pyenv-virtualenv aliases - if you use `3.7.9/envs/kolla` instead, it will be happy; though it won't with `kolla` because that gets set to a link instead of a dir.

The reason is it sets the relevant envs like this (my example with kolla-ansible@master):

  PYENV_VERSION=kolla-ansible@master
  VIRTUAL_ENV=/home/test/.pyenv/versions/3.8.5/envs/kolla-ansible@master
  PYENV_VIRTUAL_ENV=/home/test/.pyenv/versions/3.8.5/envs/kolla-ansible@master

Note pyenv works on hashed shims and it actually consumes its own env vars in them to do its runtime magic. That said, it seems to run the bash script using `PYENV_VERSION` instead of `VIRTUAL_ENV` which breaks our logic as:

  script_bin_path != virtualenv_bin_path
  script_bin_path being
  /home/test/.pyenv/versions/kolla-ansible@master/bin
  virtualenv_bin_path being
  /home/test/.pyenv/versions/3.8.5/envs/kolla-ansible@master/bin

Changed in kolla-ansible:
assignee: zengchen (zengchen228) → Radosław Piliszek (yoctozepto)
summary: - kolla-ansible 9.2.0 not work with pyenv
+ kolla-ansible does not work with pyenv-virtualenv
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/762365
Committed: https://opendev.org/openstack/kolla-ansible/commit/aaab1d1b68db0c737b63ae6bce3179df0487f23f
Submitter: Zuul
Branch: master

commit aaab1d1b68db0c737b63ae6bce3179df0487f23f
Author: zengchen228 <email address hidden>
Date: Thu Nov 12 01:11:39 2020 +0800

    Fix kolla-ansible to work with pyenv-virtualenv

    One of the pyenv-virtualenv-set-up aliases depends on a symlink.
    It seems pyenv runs the bash script from such a path and it fails
    because of a failing comparison (VIRTUAL_ENV not detected).

    The VIRTUAL_ENV is ensured to be fully resolved as well for safety.

    This requires readlink from GNU coreutils but all supported platforms
    have it by default.

    Extra comments included, as well as simplification of directory
    detection - readlink handles this (not that `bin` itself was
    ever a symlink...).

    Closes-Bug: #1903887
    Co-Authored-By: Radosław Piliszek <email address hidden>
    Change-Id: I2fe6eb13ce7be68d346b1b3b7036859f34c896c4

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/763366

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/763367

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/763368

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/763369

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/victoria)

Reviewed: https://review.opendev.org/763366
Committed: https://opendev.org/openstack/kolla-ansible/commit/7df4f403f25e21756ec7c19ac5b8df0bae36300a
Submitter: Zuul
Branch: stable/victoria

commit 7df4f403f25e21756ec7c19ac5b8df0bae36300a
Author: zengchen228 <email address hidden>
Date: Thu Nov 12 01:11:39 2020 +0800

    Fix kolla-ansible to work with pyenv-virtualenv

    One of the pyenv-virtualenv-set-up aliases depends on a symlink.
    It seems pyenv runs the bash script from such a path and it fails
    because of a failing comparison (VIRTUAL_ENV not detected).

    The VIRTUAL_ENV is ensured to be fully resolved as well for safety.

    This requires readlink from GNU coreutils but all supported platforms
    have it by default.

    Extra comments included, as well as simplification of directory
    detection - readlink handles this (not that `bin` itself was
    ever a symlink...).

    Closes-Bug: #1903887
    Co-Authored-By: Radosław Piliszek <email address hidden>
    Change-Id: I2fe6eb13ce7be68d346b1b3b7036859f34c896c4
    (cherry picked from commit aaab1d1b68db0c737b63ae6bce3179df0487f23f)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/763367
Committed: https://opendev.org/openstack/kolla-ansible/commit/f9a45abfb54215109e1fceeeff81b08625ecfd31
Submitter: Zuul
Branch: stable/ussuri

commit f9a45abfb54215109e1fceeeff81b08625ecfd31
Author: zengchen228 <email address hidden>
Date: Thu Nov 12 01:11:39 2020 +0800

    Fix kolla-ansible to work with pyenv-virtualenv

    One of the pyenv-virtualenv-set-up aliases depends on a symlink.
    It seems pyenv runs the bash script from such a path and it fails
    because of a failing comparison (VIRTUAL_ENV not detected).

    The VIRTUAL_ENV is ensured to be fully resolved as well for safety.

    This requires readlink from GNU coreutils but all supported platforms
    have it by default.

    Extra comments included, as well as simplification of directory
    detection - readlink handles this (not that `bin` itself was
    ever a symlink...).

    Closes-Bug: #1903887
    Co-Authored-By: Radosław Piliszek <email address hidden>
    Change-Id: I2fe6eb13ce7be68d346b1b3b7036859f34c896c4
    (cherry picked from commit aaab1d1b68db0c737b63ae6bce3179df0487f23f)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 10.2.0

This issue was fixed in the openstack/kolla-ansible 10.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 11.0.0.0rc2

This issue was fixed in the openstack/kolla-ansible 11.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 9.3.0

This issue was fixed in the openstack/kolla-ansible 9.3.0 release.

Mark Goddard (mgoddard)
Changed in kolla-ansible:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 12.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 12.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible stein-eol

This issue was fixed in the openstack/kolla-ansible stein-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.