python-openstackclient 6.0.0 conflicting dependencies

[Ananya Banerjee]

fs001 validations on wallaby is failing in
TASK [ovb-manage : Install openstack clients in venv]

2022-09-27 22:42:54.016838 | primary | "msg": "stdout: Looking in indexes: http://mirror.regionone.ibm-bm2-nodepool.rdoproject.org/pypi/simple, http://mirror.regionone.ibm-bm2-nodepool.rdoproject.org/wheel/centos-8-x86_64\n\nThe conflict is caused by:\n The user requested python-openstackclient\n The user requested (constraint) python-openstackclient===6.0.0\n\nTo fix this you could try to:\n1. loosen the range of package versions you've specified\n2. remove package versions to allow pip attempt to solve the dependency conflict\n\n\n:stderr: ERROR: Cannot install python-openstackclient because these package versions have conflicting dependencies.\nERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies\n"
2022-09-27 22:42:54.016869 | primary | }

https://logserver.rdoproject.org/openstack-component-validation/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-component-wallaby-validation/8acce67/job-output.txt

[chandan kumar (chkumar246)]

It is only happening on IBM cloud.

[chandan kumar (chkumar246)]

On my testproject https://review.rdoproject.org/r/c/testproject/+/45324 and other job logs.

https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-component-wallaby-validation&result=RETRY&result=RETRY_LIMIT&skip=0
- https://review.rdoproject.org/zuul/build/99ef3e0c0c6d4d718c3d52a6a24c1523
- https://review.rdoproject.org/zuul/build/3106d9b114fa4929bfbf59b554821043
- https://review.rdoproject.org/zuul/build/6ec4ed5701e54db888a8349bc9f4a67f
- https://review.rdoproject.org/zuul/build/8acce677b70841b8a5459caf5de2ba39

^^ all these builds have same issue.

[chandan kumar (chkumar246)]

After holding the node.
```
[zuul@node-0003090102 openstack]$ cd requirements/
[zuul@node-0003090102 requirements]$ git branch
* master
  stable/newton
  stable/pike
  stable/queens
  stable/rocky
  stable/stein
  stable/train
  stable/ussuri
  stable/victoria
  stable/wallaby
  stable/xena
  stable/yoga
[zuul@node-0003090102 requirements]$ pwd
/home/zuul/src/opendev.org/openstack/requirements
[zuul@node-0003090102 requirements]$

[zuul@node-0003090102 tmp]$ python3 -m venv reproducer
[zuul@node-0003090102 tmp]$ source reproducer/bin/activate
(reproducer) [zuul@node-0003090102 tmp]$ pip install python-openstackclient -c ~/src/opendev.org/openstack/requirements/upper-constraints.txt
Ignoring pkgutil-resolve-name: markers 'python_version == "3.8"' don't match your environment
Ignoring backports.zoneinfo: markers 'python_version == "3.8"' don't match your environment
Ignoring importlib-resources: markers 'python_version == "3.8"' don't match your environment
Collecting python-openstackclient===6.0.0 (from -c /home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt (line 471))
  Could not find a version that satisfies the requirement python-openstackclient===6.0.0 (from -c /home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt (line 471)) (from versions: 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.9.0, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.4.0, 2.5.0, 2.6.0, 3.0.0, 3.0.1, 3.1.0, 3.2.0, 3.2.1, 3.3.0, 3.4.0, 3.4.1, 3.5.0, 3.6.0, 3.7.0, 3.8.0, 3.8.1, 3.8.2, 3.9.0, 3.10.0, 3.11.0, 3.12.0, 3.12.1, 3.12.2, 3.13.0, 3.14.0, 3.14.1, 3.14.2, 3.14.3, 3.15.0, 3.16.0, 3.16.1, 3.16.2, 3.16.3, 3.17.0, 3.18.0, 3.18.1, 3.19.0, 4.0.0, 4.0.1, 4.0.2, 5.0.0, 5.1.0, 5.2.0, 5.2.1, 5.2.2, 5.3.0, 5.3.1, 5.4.0, 5.5.0, 5.5.1, 5.6.0, 5.7.0, 5.8.0)
No matching distribution found for python-openstackclient===6.0.0 (from -c /home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt (line 471))
You are using pip version 9.0.3, however version 22.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(reproducer) [zuul@node-0003090102 tmp]$
```

and after checking out stable/wallaby in openstack/requirements project
```
(reproducer) [zuul@node-0003090102 tmp]$ cd ~/src/opendev.org/openstack/requirements/
(reproducer) [zuul@node-0003090102 requirements]$ git checkout stable/wallaby
Switched to branch 'stable/wallaby'
(reproducer) [zuul@node-0003090102 requirements]$ pip install python-openstackclient -c ~/src/opendev.org/openstack/requirements/upper-constraints.txt
Ignoring scipy: markers 'python_version == "3.8"' don't match your environment
Ignoring autobahn: markers 'python_version == "3.8"' don't match your environment
Ignoring numpy: markers 'python_version == "3.8"' don't match your environment
Collecting python-openstackclient===5.5.1 (from -c /home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt (line 463))
  Downloading http://mirror.regionone.ibm-bm2-nodepool.rdoproject.org/pypifiles/packages/fb/21/e3de4025d87da0dcc0441c9acc37079880669f42c733b6a965a94...

[chandan kumar (chkumar246)]

By taking a look at job defintion
https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/component-jobs-wallaby.yaml#L281
```
- job:
    name: periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-component-wallaby-validation
    parent: periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-component-master-validation
    description: |
      VF OVB job for wallaby branch. Inheriting from the master job and overriding release
      specifying parameters.
    vars:
      release: wallaby
      branch_override: 'stable/wallaby'
```
override-checkout is not defined at this job level, that's why openstack/requirements project is checkedout from master.

[chandan kumar (chkumar246)]

Another corner case, stable/wallaby uc does not work with train jobs.

[Ananya Banerjee (frenzyfriday)]

Fix: https://review.rdoproject.org/r/c/config/+/45379
tp: https://code.engineering.redhat.com/gerrit/c/testproject/+/430001 - fs35 internal are not hitting retry limit any more