Unable to stack on fedora34 due to cinder pulling in oslo.vmware and suds-jurko that uses use_2to3 that is invalid with setuptools >=58.0.0

[Lee Yarwood]

https://zuul.opendev.org/t/openstack/builds?job_name=devstack-platform-fedora-latest

https://zuul.opendev.org/t/openstack/build/3e803930a05e4fa28b3873364ec812c7

2021-10-06 12:51:30.359191 | controller | Successfully installed setuptools-58.2.0
[..]
2021-10-06 12:54:15.950378 | controller | Using python 3.9 to install /opt/stack/cinder
2021-10-06 12:54:15.952758 | controller | + inc/python:pip_install:193 : sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.9 -m pip install -c /opt/stack/requirements/upper-constraints.txt -e /opt/stack/cinder
2021-10-06 12:54:16.358711 | controller | WARNING: Running pip install with root privileges is generally not a good idea. Try `python3.9 -m pip install --user` instead.
2021-10-06 12:54:16.944883 | controller | Looking in indexes: https://mirror.iad3.inmotion.opendev.org/pypi/simple, https://mirror.iad3.inmotion.opendev.org/wheel/fedora-34-x86_64
2021-10-06 12:54:16.969945 | controller | Obtaining file:///opt/stack/cinder
[..]
2021-10-06 12:54:23.033422 | controller | Collecting suds-jurko>=0.6
2021-10-06 12:54:23.036449 | controller | Downloading https://mirror.iad3.inmotion.opendev.org/pypifiles/packages/d0/88/f3bed9b494e0dae26bd55e5f3d527b8244208125024727267e8109956a11/suds-jurko-0.6.zip (255 kB)
2021-10-06 12:54:23.411290 | controller | ERROR: Command errored out with exit status 1:
2021-10-06 12:54:23.411331 | controller | command: /usr/bin/python3.9 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z165r4bu/suds-jurko_4a7d3f7477c142668c68d48b61f4850d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z165r4bu/suds-jurko_4a7d3f7477c142668c68d48b61f4850d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-qflx0nam
2021-10-06 12:54:23.411341 | controller | cwd: /tmp/pip-install-z165r4bu/suds-jurko_4a7d3f7477c142668c68d48b61f4850d/
2021-10-06 12:54:23.411349 | controller | Complete output (1 lines):
2021-10-06 12:54:23.411356 | controller | error in suds-jurko setup command: use_2to3 is invalid.
2021-10-06 12:54:23.411363 | controller | ----------------------------------------
2021-10-06 12:54:23.411555 | controller | WARNING: Discarding https://mirror.iad3.inmotion.opendev.org/pypifiles/packages/d0/88/f3bed9b494e0dae26bd55e5f3d527b8244208125024727267e8109956a11/suds-jurko-0.6.zip#sha256=1cb7252cb13018fc32887c3a834ed7c6648a5b5c4c159be5806da2e1785399e8 (from https://mirror.iad3.inmotion.opendev.org/pypi/simple/suds-jurko/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
2021-10-06 12:54:23.428057 | controller | Downloading https://mirror.iad3.inmotion.opendev.org/pypifiles/packages/bd/6f/54fbf0999a606680d27c69b1ad12dfff62768ecb9fe48524cebda6eb4423/suds-jurko-0.6.tar.bz2 (143 kB)
2021-10-06 12:54:23.884635 | controller | ERROR: Command errored out with exit status 1:
2021-10-06 12:54:23.884668 | controller | command: /usr/bin/python3.9 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z165r4bu/suds-jurko_3e1b1f61ad89431f989437799fa47041/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z165r4bu/suds-jurko_3e1b1f61ad89431f989437799fa47041/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-cz87bqfy
2021-10-06 12:54:23.884675 | controller | cwd: /tmp/pip-install-z165r4bu/suds-jurko_3e1b1f61ad89431f989437799fa47041/
2021-10-06 12:54:23.884681 | controller | Complete output (1 lines):
2021-10-06 12:54:23.884686 | controller | error in suds-jurko setup command: use_2to3 is invalid.
[..]
2021-10-06 12:54:31.011976 | controller | ERROR: Could not find a version that satisfies the requirement suds-jurko>=0.6 (from oslo-vmware)
2021-10-06 12:54:31.012096 | controller | ERROR: No matching distribution found for suds-jurko>=0.6

[Lee Yarwood (lyarwood)]

https://codesearch.opendev.org/?q=oslo.vmware&i=nope&literal=nope&files=.*requirements.*&excludeFiles=&repos=

So Cinder and a few other projects list oslo.vmware as a requirement. It's interesting that Nova doesn't and only lists it as a test-requirement even with the vmware driver still in tree.

[Dr. Jens Harbott (j-harbott)]

designate pulls in suds-jurko directly and is thus also affected.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/designate/+/813151

[Lingxian Kong (kong)]

I can reproduce this bug on Ubuntu 20.04 cloud image.

[Gunju Kim (gjkim)]

I think this issue has occurred since https://opendev.org/openstack/requirements/commit/1ac14394617d226808d3ce5adf01d93f874cc1ba.

I am not familiar with python packaging.
IMHO, there are some workarounds
1. pin setuptools to 58.0.0 or lower (This works for me.)
2. replace all suds-jurko dependencies with suds-community as suds-jurko has not been maintained long ago.

[Gunju Kim (gjkim)]

Here is a related proposal https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Dr. Jens Harbott <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/designate/+/813151
Reason: Superseded by https://review.opendev.org/c/openstack/designate/+/813380

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/designate/+/813380
Committed: https://opendev.org/openstack/designate/commit/771197c2f31209fe135888aef6f2828af4889653
Submitter: "Zuul (22348)"
Branch: master

commit 771197c2f31209fe135888aef6f2828af4889653
Author: Gunju Kim <email address hidden>
Date: Mon Oct 11 15:01:45 2021 +0900

    Remove Akamai (eDNS SOAP API) backend

    The Akamai eDNS SOAP API this backend uses no longer exists at Akamai.

    Related-Bug: 1946340
    Change-Id: I6545781f263c6a3d124364785aedcf2518116485

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/oslo.vmware/+/813377
Committed: https://opendev.org/openstack/oslo.vmware/commit/7bc343173751330d2e989c28bc6fefed56d358d6
Submitter: "Zuul (22348)"
Branch: master

commit 7bc343173751330d2e989c28bc6fefed56d358d6
Author: Gunju Kim <email address hidden>
Date: Mon Oct 11 14:49:33 2021 +0900

    Replace suds-jurko with suds-community

    suds-jurko is not compatible with setuptools >58.0.0 and has not been
    maintained for a long time. Let's use suds-community instead.

    Related-Bug: 1946340
    Depends-On: I4eb235c3ad0f1296fc81a9b98d7e687ac9923b59
    Change-Id: Ie3f317974d138be4050d0ce196bcc000494a0eee

[Tom Barron (tpb)]

For cinder build in devstack we also need a tagged release that includes the change merged in 813377, plus an upper_constraints change. This review [1] will do the release, and automation should generate the corresponding requirements review.

https://review.opendev.org/c/openstack/releases/+/814012

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/814174

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/814174
Committed: https://opendev.org/openstack/cinder/commit/fd5ec459c8d51ecb76ca680e0993726e647ce406
Submitter: "Zuul (22348)"
Branch: master

commit fd5ec459c8d51ecb76ca680e0993726e647ce406
Author: Brian Rosmaita <email address hidden>
Date: Fri Oct 15 10:15:56 2021 -0400

    Increase min version of oslo.vmware to 3.10.0

    Older versions have a dependency that is no longer satisfiable
    on all supported test platforms. See discussion on I5925d1d8cbaad6
    and bug #1946340.

    Depends-on: I2a06f0d302325425e67f4ca3dcca14fd73d41c3e

    Change-Id: Icfe49cd2303b5d7091bc1f24c09bde0bc624ab6a
    Related-bug: #1946340

[Michael Johnson (johnsom)]

Designate has removed the requirement for suds-jurko in this patch: https://review.opendev.org/c/openstack/designate/+/813380

[Dr. Jens Harbott (j-harbott)]

Afaict this has been resolved in all affected projects now.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/oslo.vmware/+/835672

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/oslo.vmware/+/835672
Committed: https://opendev.org/openstack/oslo.vmware/commit/bfda4c7fa96dc063c419e78565a6500867b9be79
Submitter: "Zuul (22348)"
Branch: stable/xena

commit bfda4c7fa96dc063c419e78565a6500867b9be79
Author: Gunju Kim <email address hidden>
Date: Mon Oct 11 14:49:33 2021 +0900

    Replace suds-jurko with suds-community

    suds-jurko is not compatible with setuptools >58.0.0 and has not been
    maintained for a long time. Let's use suds-community instead.

    Related-Bug: 1946340
    Change-Id: Ie3f317974d138be4050d0ce196bcc000494a0eee
    (cherry picked from commit 7bc343173751330d2e989c28bc6fefed56d358d6)