Tempest installation is failing with tox4 allowlist_externals changes

[Ghanshyam Mann]

With tox4 changes which are more strict on allowlist_externals and tempest recently changed allowlist_externals not to be * - https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

++ lib/tempest:configure_tempest:710 : rm -f /tmp/tempest_u_c_m.AYJGDlC9cW
++ lib/tempest:configure_tempest:713 : [[ False == \T\r\u\e ]]
++ lib/tempest:configure_tempest:721 : [[ True == \F\a\l\s\e ]]
++ lib/tempest:configure_tempest:725 : iniset /opt/stack/tempest/etc/tempest.conf auth use_dynamic_credentials True
++ lib/tempest:configure_tempest:731 : tox -evenv -- tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
venv: install_deps> python -I -m pip install -r /opt/stack/tempest/requirements.txt -r /opt/stack/tempest/doc/requirements.txt -c https://releases.openstack.org/constraints/upper/master
venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
venv: failed with tempest is not allowed, use allowlist_externals to allow it

One obvious bug is verify-config in devstack lib/tempest not using the venv-tempest where tempest is installed.

[Ghanshyam Mann (ghanshyammann)]

Stable branch jobs (grenade on master) are running on Ubuntu focal (and focal job on master) which are picking the tox 4.0.2 from python3.8[1] and other master job running on Ubuntu Jammy picking the
tox 3.27.1 from python3.10[2]

[1] /usr/local/lib/python3.8/dist-packages (4.0.2)

[2] /usr/local/lib/python3.10/dist-packages (3.27.1)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/devstack/+/867065

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/devstack/+/867066

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/devstack/+/867067

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/devstack/+/867068

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/devstack/+/867069

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/devstack/+/867070

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/devstack/+/867071

[Ghanshyam Mann (ghanshyammann)]

let's pin the tox to be <4.0.0 for stable branch testing unless anyone wants to test/fix all stable branch jobs with a new tox.

But on master, we should fix and use tox 4.0.0.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867071
Committed: https://opendev.org/openstack/devstack/commit/b11a403c37f9873cdaaa6afc31619ab72b0fd8c2
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit b11a403c37f9873cdaaa6afc31619ab72b0fd8c2
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
    (cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)
    (cherry picked from commit 395f44708506b30b29b35dd655dddd434bac9a08)
    (cherry picked from commit 02f286a80a478d662e079fe066d311e4cf7264d8)
    (cherry picked from commit 0a72476d895d925fdd37b4e42e2930b415b99d8f)
    (cherry picked from commit 02b6981f5121293840cb82cc06a28ba71b2ca28b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867070
Committed: https://opendev.org/openstack/devstack/commit/02b6981f5121293840cb82cc06a28ba71b2ca28b
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 02b6981f5121293840cb82cc06a28ba71b2ca28b
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
    (cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)
    (cherry picked from commit 395f44708506b30b29b35dd655dddd434bac9a08)
    (cherry picked from commit 02f286a80a478d662e079fe066d311e4cf7264d8)
    (cherry picked from commit 0a72476d895d925fdd37b4e42e2930b415b99d8f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867069
Committed: https://opendev.org/openstack/devstack/commit/d4b040ed3329755d76b26d086439f1fc5ded215b
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit d4b040ed3329755d76b26d086439f1fc5ded215b
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/867070

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
    (cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)
    (cherry picked from commit 395f44708506b30b29b35dd655dddd434bac9a08)
    (cherry picked from commit 02f286a80a478d662e079fe066d311e4cf7264d8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867068
Committed: https://opendev.org/openstack/devstack/commit/1c064ef80afa44ba096a3a6ac6130b64662d50c8
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 1c064ef80afa44ba096a3a6ac6130b64662d50c8
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/867069

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
    (cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)
    (cherry picked from commit 395f44708506b30b29b35dd655dddd434bac9a08)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867067
Committed: https://opendev.org/openstack/devstack/commit/9e73a67417fd05deda0ed6b0505f4fbbe35b11a5
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 9e73a67417fd05deda0ed6b0505f4fbbe35b11a5
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/867068

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)
    (cherry picked from commit a3227ba0c0a6b951b301e556f78e634e6ce9e770)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867066
Committed: https://opendev.org/openstack/devstack/commit/211cc4c036b6cb13598b87e6e3bbc3c74538a902
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 211cc4c036b6cb13598b87e6e3bbc3c74538a902
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/867067

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b
    (cherry picked from commit ba54baa4253f2c770760b59a4a8f28317e5e260d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/867065
Committed: https://opendev.org/openstack/devstack/commit/9a1be7794bd3b1b06a89183a800f42f77cd1b1b9
Submitter: "Zuul (22348)"
Branch: master

commit 9a1be7794bd3b1b06a89183a800f42f77cd1b1b9
Author: Ghanshyam Mann <email address hidden>
Date: Thu Dec 8 20:24:46 2022 -0600

    Pin tox<4.0.0 for <=stable/zed branch testing

    Tox 4.0.0 has some incompatible changes, epecially more
    strict on allowlist_externals. Tempest recently changed
    allowlist_externals not to be *[1] causing the failure
    on jobs where lib/tempest failing to run the tempest
    as command in virtual env.

    ----------
    venv: commands[0]> tempest verify-config -uro /tmp/tmp.qH5KgJHTF4
    venv: failed with tempest is not allowed, use allowlist_externals to allow it
    ------

    We do not need to test/fix the <=stable/zed branches with tox 4.0.0
    and pinning them with the compatible tox version of the time stable
    brnaches were releaased is better way.

    This commit proposes:
    1. Pinning the tox<4.0.0 for <=stable/ze branches testing
    2. Workaround to unblock the master gate by pinning it <4.0.0 but
    we should make our testing compatible with tox 4.0.0 soon.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/867066

    Related-Bug: #1999183

    [1] https://review.opendev.org/c/openstack/tempest/+/865314 devstack based job started failing to run tempest command on venv.

    Change-Id: I9a138af94dedc0d8ce5a0d519d75779415d3c30b

[Ghanshyam Mann (ghanshyammann)]

changing its priority as we have workaround of pining tox<4 on master devstack jobs which we should unpin but that need investigation on tempest virtual env issue.