AttributeError: 'NoneType' object has no attribute 'version_tuple

[chandan kumar]

On CS9 pipeline, Container Build and Image build jobs are failing with following errors:

```
TASK [oooci-build-images : Create temp venv w/ virtualenv command]
2021-12-02 10:32:01.437898 | primary | ERROR
2021-12-02 10:32:01.438246 | primary | {
2021-12-02 10:32:01.438304 | primary | "msg": "stdout: AttributeError: 'NoneType' object has no attribute 'version_tuple'\n"
2021-12-02 10:32:01.438348 | primary | }
```
OR
```
2021-12-02 09:17:04.738180 | LOOP [build-containers : Install deps from upper-constraints]
2021-12-02 09:17:06.197455 | primary | ERROR: Item: tripleo-common
2021-12-02 09:17:06.197824 | primary | {
2021-12-02 09:17:06.197887 | primary | "ansible_loop_var": "item",
2021-12-02 09:17:06.197953 | primary | "item": "tripleo-common",
2021-12-02 09:17:06.198004 | primary | "msg": "stdout: AttributeError: 'NoneType' object has no attribute 'version_tuple'\n"
2021-12-02 09:17:06.198071 | primary | }
2021-12-02 09:17:06.198157 | primary | ok: All items complete
2021-12-02 09:17:06.198207 |
2021-12-02 09:17:07.300107 | primary | ERROR: Item: python-tripleoclient
2021-12-02 09:17:07.300578 | primary | {
2021-12-02 09:17:07.300664 | primary | "ansible_loop_var": "item",
2021-12-02 09:17:07.300702 | primary | "item": "python-tripleoclient",
2021-12-02 09:17:07.300731 | primary | "msg": "stdout: AttributeError: 'NoneType' object has no attribute 'version_tuple'\n"
2021-12-02 09:17:07.300759 | primary | }
2021-12-02 09:17:08.270688 | primary | ERROR: Item: tripleo-ansible
2021-12-02 09:17:08.270891 | primary | {
2021-12-02 09:17:08.270930 | primary | "ansible_loop_var": "item",
2021-12-02 09:17:08.270959 | primary | "item": "tripleo-ansible",
2021-12-02 09:17:08.270986 | primary | "msg": "stdout: AttributeError: 'NoneType' object has no attribute 'version_tuple'\n"
2021-12-02 09:17:08.271025 | primary | }
```

By Looking at both the failed tasks:

https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/oooci-build-images/tasks/pre.yaml#L100

```
virtualenv_command: "{{ 'virtualenv' if ansible_distribution_major_version is version('9', '==') else 'virtualenv-3' }}"
```

and
https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/venv_setup.yml#L21

```
virtualenv_python: "{{ virtualenv_python[ansible_distribution_major_version|int] }}"
```

In both cases, virtualenv-3 might be available.

Better we can switch to python -m venv as a virtualenv command to avoid this issue.

Logs:
[1]. https://logserver.rdoproject.org/01/37001/1/check/periodic-tripleo-centos-9-buildimage-ironic-python-agent-master/2e8ac25/job-output.txt

[2]. https://logserver.rdoproject.org/97/36997/3/check/periodic-tripleo-ci-build-containers-centos-9-push-master/4b0d576/job-output.txt

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/820172

[chandan kumar (chkumar246)]

[root@2134a04ff981 yum.repos.d]# dnf whatprovides virtualenv-3
Last metadata expiration check: 0:03:58 ago on Thu Dec 2 15:40:52 2021.
Error: No Matches found
[root@2134a04ff981 yum.repos.d]# cat /etc/redhat-release
CentOS Stream release 9
[root@2134a04ff981 yum.repos.d]# dnf whatprovides virtualenv
Last metadata expiration check: 0:04:10 ago on Thu Dec 2 15:40:52 2021.
python3-virtualenv-20.4.4-1.el9s.noarch : Tool to create isolated Python environments
Repo : @System
Matched from:
Provide : virtualenv = 20.4.4-1.el9s

python3-virtualenv-20.4.4-1.el9s.noarch : Tool to create isolated Python environments
Repo : delorean-master-build-deps
Matched from:
Provide : virtualenv = 20.4.4-1.el9s

[root@2134a04ff981 yum.repos.d]#

[Marios Andreou (marios-b)]

the patch from chkumar at https://review.opendev.org/c/openstack/tripleo-ci/+/820172/ fixes the issue for centos9 - green runs with https://review.rdoproject.org/r/c/testproject/+/36817/10#message-59f74de48ce71516dae4ccd64d60a312432b279e

however it breaks the c8 builds as seen upstream with https://review.opendev.org/c/openstack/tripleo-ci/+/820172/7#message-5a3cf6f3543b9261eddee6ed2f69b579ab3c2861

I am digging there we need to fix it before we can merge tripleo-ci/+/820172

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ci/+/820172
Committed: https://opendev.org/openstack/tripleo-ci/commit/caa8675adcc839909ff9290daa7a502fe89db291
Submitter: "Zuul (22348)"
Branch: master

commit caa8675adcc839909ff9290daa7a502fe89db291
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Thu Dec 2 18:33:31 2021 +0530

    Use virtualenv_command as ensure_pip_virtualenv_command

    In tripleo-ci, oooci-build-images[1] and build-containers[2]
    roles, virtualenv_python and virtualenv_command points to
    virtualenv-3.

    On EL9, pip module is throwing error:
    ```
    AttributeError: 'NoneType' object has no attribute 'version_tuple'
    ```

    Switching to ensure_pip_virtualenv_command var
    (comes from ensure_pip role) fixes the issue.

    Python3 is already available on CS7,8 & 9.
    It also removes the hacks related to python-virtualenv and its
    installation.

    Closes-Bug: #1953032
    Closes-Bug: #1745626

    Co-Authored-By: Marios Andreou (marios) <email address hidden>
    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: I6de99ca4d6be46960b6061225e4a16c76edc8455

[Marios Andreou (marios-b)]

holding for a bit... we have some similar errors in tempest now - see comment @ https://review.rdoproject.org/r/c/testproject/+/36508/14#message-d6805f1ee65e267381df58b4ef2cf26cc1a77706

from https://logserver.rdoproject.org/08/36508/14/check/periodic-tripleo-ci-centos-9-scenario007-standalone-master/ebf137b/job-output.txt

2021-12-08 04:12:22.790336 | primary | TASK [python_venv_build : Create the virtualenv (if it does not exist)] ********
2021-12-08 04:12:22.790404 | primary | Wednesday 08 December 2021 04:12:22 -0500 (0:00:01.716) 0:31:47.929 ****
2021-12-08 04:12:24.791707 | primary | fatal: [undercloud]: FAILED! => {"changed": true, "cmd": ["virtualenv", "--no-download", "--python=python3", "/home/zuul/stackviz_venv"], "delta": "0:00:00.302069", "end": "2021-12-08 09:12:24.522950", "msg": "non-zero return code", "rc": 1, "start": "2021-12-08 09:12:24.220881", "stderr": "", "stderr_lines": [], "stdout": "AttributeError: 'NoneType' object has no attribute 'version_tuple'", "stdout_lines": ["AttributeError: 'NoneType' object has no attribute 'version_tuple'"]}
2021-12-08 04:12:24.793670 | primary |

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/821012

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/821012
Committed: https://opendev.org/openstack/tripleo-quickstart-extras/commit/0241256ad0ad777999b6b5588ffa3d01a798acf3
Submitter: "Zuul (22348)"
Branch: master

commit 0241256ad0ad777999b6b5588ffa3d01a798acf3
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Wed Dec 8 17:37:02 2021 +0530

    Disable Stackviz for CS9

    CS9 jobs are failing with following error
    ```
    'NoneType' object has no attribute 'version_tuple'
    ```
    while installing stackviz via python_venv_build role.

    The python_venv_build role uses virtualenv binary not
    python -m venv utility. Moving to python -m venv might
    break the role.

    Till we get a proper fix, we are disabling stackviz
    installation on CS9.

    Related-Bug: #1953032

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: Ic5bc9a5b4be36b62ce60d6db362821ea4da2239a

[Marios Andreou (marios-b)]

I am not sure what to do with this bug now... the workaround/disable stackviz for cs9 merged @ https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/821012

I'm going to move to fix-released as it no longer blocks us and because otherwise it will likely stay as 'In Progress' for a long time...

There is a comment in the code at https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/d3d788717cf5560656ea75f9b80dc43bcea713d0/playbooks/tasks/tempest.yml#L85-L87 so when we revisit we can update this bug again.