tempest 28.0.0 cap for stable/train mismatch the constraints and re-create venv for tempest plugins tests

[Ghanshyam Mann]

This issue came up before also during we cap the Tempest version for stable branch tests(the one moved to EM state).

Tempest use the tox constraints file while creating the tempest virtual env in devstack/lib/tempest, installing the tempest plugins and running the tests like in run-tempest ansible roles. If anywhere tox constraints file is used differently then venv will be recreated and installed tempest plugins will be gone from that recreated venv so tempest will not be able to run the plugins tests and fail job.

It was fixed by using the same tox constraint file we used in devstack/lib/tempest in run-tempest role so that tempest venv does not get re-created, example: https://review.opendev.org/c/openstack/tempest/+/774060

and then I tried to make it generic by fetching the devstack localrc - https://review.opendev.org/c/openstack/tempest/+/774913 But this solution does not seems working as we can see it is skipped in current stable/train testing with tempest 28.0.0 - https://zuul.opendev.org/t/openstack/build/8a010a44d3bb43838901a6f025fd96e5/console

I need to debug more why ansible roles fetching the devstack localrc not working but meanwhile we need to fix it in old way to unblock the tempest plugin gate.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tempest/+/822339

[Ghanshyam Mann (ghanshyammann)]

Fixes:
- https://review.opendev.org/c/openstack/devstack/+/822380
- https://review.opendev.org/c/openstack/tempest/+/822339
- these are testing patches #link https://review.opendev.org/q/topic:%22tempest-train-last%22+(status:open%20OR%20status:merged)

[Ghanshyam Mann (ghanshyammann)]

Trying tempest 26.1.0 with workaround on new args with old stestr:

https://review.opendev.org/q/topic:%22train-tempest-cap%22+(status:open%20OR%20status:merged)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tempest/+/822339
Committed: https://opendev.org/openstack/tempest/commit/5ff5dabd6e6bafde1a68e70f9a6869cee1753850
Submitter: "Zuul (22348)"
Branch: master

commit 5ff5dabd6e6bafde1a68e70f9a6869cee1753850
Author: Ghanshyam Mann <email address hidden>
Date: Mon Dec 20 10:07:49 2021 -0600

    Fix stable/train testing with tempest 26.1.0

    Tis issue came up before also during we cap the Tempest
    version for stable branch tests(the one moved to EM state).

    Tempest use the tox constraints file while creating the tempest
    virtual env in devstack/lib/tempest, installing the tempest
    plugins and running the tests like in run-tempest ansible roles.
    If anywhere tox constraints file is used differently then venv
    will be recreated and installed tempest plugins will be gone from
    that recreated venv so tempest will not be able to run the plugins
    tests and fail job.

    Current failure: https://aca43ed1a01d95dea0ee-dcdb6cbb330bdac08ffee1284f86c919.ssl.cf2.rackcdn.com/822225/3/check/octavia-v2-dsvm-scenario/f60e4d3/job-output.txt

    Earlier I tried to fix it in a generic way by fetching the devstack
    localrc - https://review.opendev.org/c/openstack/tempest/+/774913
    but this solution does not seems working as we can see it is skipped
    in current stable/train testing with tempest 26.1.0

    Adding stable/train explicitly in list for using the stable constraint
    in run-tempest role also which will be in sync with what devstack/lib/tempest
    use. And later I will debug why generic way is not working.

    Because of Tempest 26.1.0 and with stestr 2.5.1 (beacause of upper
    constraints of stestr 2.5.1 in stable/train) which does not have
    new args exclude-regex so let's fallback to old arg if new arg is passed.

    Depends-On: https://review.opendev.org/c/openstack/devstack/+/822380
    Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/825708

    Closes-Bug: #1955418
    Change-Id: I105c839c38adfba1f9e22123ec3a99ffc0b5b72e

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 30.0.0 release.