pip 20.3 break devstack: ERROR: Links are not allowed as constraints

[Ghanshyam Mann]

With pip 20.3 is released (https://pip.pypa.io/en/stable/news/#id1) devstack started failing during keystone install with:

2020-11-30 15:14:31.117 | + inc/python:pip_install:193 :
sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy=
https_proxy= no_proxy= PIP_FIND_LINKS=
SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.6 -m pip install -c
/opt/stack/old/requirements/upper-constraints.txt -e
/opt/stack/old/neutron
2020-11-30 15:14:32.271 | Looking in indexes:
https://mirror.gra1.ovh.opendev.org/pypi/simple,
https://mirror.gra1.ovh.opendev.org/wheel/ubuntu-18.04-x86_64
2020-11-30 15:14:32.272 | DEPRECATION: Constraints are only allowed to
take the form of a package name and a version specifier. Other forms
were originally permitted as an accident of the implementation, but
were undocumented. The new implementation of the resolver no longer
supports these forms. A possible replacement is replacing the
constraint with a requirement.. You can find discussion regarding this
at https://github.com/pypa/pip/issues/8210.
2020-11-30 15:14:32.272 | ERROR: Links are not allowed as constraints

Logstash signature:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Links%20are%20not%20allowed%20as%20constraints%5C%22

It seems not all the jobs are effected but at least grenade on master
is hit [3]. It seems that passing jobs are using pip from the base
image instead of installing a fresh one from pypi.

https://zuul.opendev.org/t/openstack/builds?job_name=grenade&branch=master

[Ghanshyam Mann (ghanshyammann)]

Workaround to cap pip <23.0

 https://review.opendev.org/q/topic:%22cap-pip-23-0%22+(status:open%20OR%20status:merged)

[Elod Illes (elod-illes)]

The error message says:

"DEPRECATION: Constraints are only allowed to
take the form of a package name and a version specifier. Other forms
were originally permitted as an accident of the implementation, but
were undocumented. The new implementation of the resolver no longer
supports these forms. A possible replacement is replacing the
constraint with a requirement.."

So as I understand the only way is to use the links only as requirement
(links in costraint is completely unsupported).

Currently we are replacing the constraints of packages to *links*, for
those packages which are installed from source [1], to avoid that an
install pulls that package from pypi instead of having it from source.
This link injection causes the above deprecation message.

I've experimented a bit and found that it is enough to just simply
filter out these packages from the upper-constraints.txt. This can be
done because we install the packages from source right after the
constraints file editing (at least this is what I experienced). If I'm
not mistaken, this way, if there is a package dependency that is
available from source, then it is installed already and won't be
reinstalled.

One issue might happen, though, with the following scenario:
* we have two package, both from source
* one depends on the other
* we use the depends-on tag at the package, which is the dependency of
  the other package
In this case we *might* reinstall the package from pypi... but I'm not
sure.

Anyway, I've uploaded some Work In Progress patch [2], which shows that
the filtering is enough, as far as I see. In this way we can use latest
pip, as we are not using links anymore.

Any opinion about this idea and possible drawbacks?

[1] https://opendev.org/openstack/devstack/src/commit/5dff8186027cf109369d20a57cdadd50a73fac74/inc/python#L381-L386
[2] https://review.opendev.org/q/I6929b0f971ea72e5d7b77d85219db069431c48d3

[Martin Kopec (mkopec)]

any updates?
seems like it's reproducible even with pip==20.2.4 now -> https://bugs.launchpad.net/tempest/+bug/1912178

[Geoffroy Van Cutsem (gvancuts)]

I'm hitting this as well, on devstack stable/train, Ubuntu 18.04 (running in an LXC container).

I've tried the workaround that was merged, i.e. to modify `tools/cap-pip.txt` to include "pip<20.3" but the error persists. I also tried to make that "pip<20.2.4" (see also comment above) but that did not help either.

Any idea how to resolve this?

[Richard Evans (revans-cymycloud)]

Same issue.

++lib/tempest:install_tempest:747 /opt/stack/tempest/.tox/tempest/bin/pip install -c /opt/stack/requirements/upper-constraints.txt -r requirements.txt
DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210.
ERROR: Links are not allowed as constraints

/opt/stack/requirements/upper-constraints.txt File contains many "-e file:///opt/stack/" links. Is there a branch to check out that resolves this?

[Eduardo Santos (ecsantos)]

Reproduced on stable/train, Ubuntu 18.04. I believe the current fix is only available down to Ussuri, right? Modifying cap-pip.txt to lower pip versions also didn't work.

[Jesse Quinn (jessequinn)]

same issue.

pip 21.0.1 from /opt/stack/.local/lib/python3.8/site-packages/pip (python 3.8)
VERSION="20.04.2 LTS (Focal Fossa)"

victoria branch.

DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210.
ERROR: Links are not allowed as constraints

[ldin (ldin)]

Has anyone finally been able to find a solution?

[Spencer (spencerharmon)]

I'm looking for a solution for this bug also. Ubuntu 18.04. stable/stein

[Radosław Piliszek (yoctozepto)]

Likely correct fix: https://review.opendev.org/c/openstack/devstack/+/802642

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/802642
Committed: https://opendev.org/openstack/devstack/commit/6b9a5646225a766f6240e2a1a93a92b82e088aa0
Submitter: "Zuul (22348)"
Branch: master

commit 6b9a5646225a766f6240e2a1a93a92b82e088aa0
Author: Ian Wienand <email address hidden>
Date: Wed Jul 28 11:19:57 2021 +1000

    Revert "Workaround for new pip 20.3 behavior"

    This reverts commit 7a3a7ce876a37376fe0dca7278e41a4f46867daa and
    bcd0acf6c0b5d6501e91133c3a937b3fc40f7122 and part of
    f1ed7c77c50ac28cb58c9f7ed885c6a3e0a75403 which all cap our pip
    installs.

    Given the pip ecosystem can often incorporate major changes, tracking
    upstream at least generally gives us one problem at a time to solve
    rather than trying to handle version jumps when LTS distros update.

    The new dependency resolver included some changes that disallow
    setting URL's like "file:///path/to/project#egg=project" in
    constraints. Apparently the fact it used to work was an accident of
    the requires/constraints mechanism; it does make some sense as the URL
    doesn't really have a version-number that the resolver can put in an
    ordering graph.

    The _setup_package_with_constraints_edit function comment highlights
    what this is trying to do

     # Updates the constraints from REQUIREMENTS_DIR to reflect the
     # future installed state of this package. This ensures when we
     # install this package we get the from source version.

    In other words; if constraints has "foo==1.2.3" and Zuul has checked
    out "foo" for testing, we have to make sure pip doesn't choose version
    1.2.3 from pypi.

    It seems like removing the entry from upper-requirements.txt is the
    important part; adding the URL path to the on-disk version was just
    something that seemed to work at the time, but isn't really necessary.
    We will install the package in question which will be the latest
    version (from Zuul checkout) and without the package in
    upper-requirements.txt nothing will try and downgrade it.

    Therefore the solution proposed here is to remove the adding of the
    URL parts.

    This allows us to uncap pip and restore testing with the new
    dependency resolver.

    Closes-Bug: #1906322
    Change-Id: Ib9ba52147199a9d6d0293182d5db50c4a567d677

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/devstack/+/805008

[Wenping Song (wenping1)]

still error under pip==21.2.4.
with the message "Editable requirements are not allowed as constraints".

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Dr. Jens Harbott <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/devstack/+/765149
Reason: I think this has been implemented as part of https://review.opendev.org/c/openstack/devstack/+/802642

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rabi Mishra <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/devstack/+/805008
Reason: Did not see any interest to merge this.

[melanie witt (melwitt)]

Looks like this bug has cropped up on stable/wallaby today, seeing this error on failing grenade jobs [1][2]:

022-06-01 18:33:39.549 | full installdeps: -c/opt/stack/new/requirements/upper-constraints.txt, -r/opt/stack/new/tempest/requirements.txt
2022-06-01 18:33:41.439 | ERROR: invocation failed (exit code 1), logfile: /opt/stack/new/tempest/.tox/tempest/log/full-1.log
2022-06-01 18:33:41.439 | ================================== log start ===================================
2022-06-01 18:33:41.439 | Looking in indexes: https://mirror.mtl01.iweb.opendev.org/pypi/simple, https://mirror.mtl01.iweb.opendev.org/wheel/ubuntu-20.04-x86_64
2022-06-01 18:33:41.439 | DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210.
2022-06-01 18:33:41.439 | ERROR: Links are not allowed as constraints
2022-06-01 18:33:41.439 | WARNING: You are using pip version 21.0.1; however, version 22.1.2 is available.
2022-06-01 18:33:41.439 | You should consider upgrading via the '/opt/stack/new/tempest/.tox/tempest/bin/python -m pip install --upgrade pip' command.
2022-06-01 18:33:41.439 |
2022-06-01 18:33:41.439 | =================================== log end ====================================
2022-06-01 18:33:41.440 | ERROR: could not install deps [-c/opt/stack/new/requirements/upper-constraints.txt, -r/opt/stack/new/tempest/requirements.txt]; v = InvocationError('/opt/stack/new/tempest/.tox/tempest/bin/pip install -c/opt/stack/new/requirements/upper-constraints.txt -r/opt/stack/new/tempest/requirements.txt', 1)
2022-06-01 18:33:41.440 | ___________________________________ summary ____________________________________
2022-06-01 18:33:41.440 | ERROR: full: could not install deps [-c/opt/stack/new/requirements/upper-constraints.txt, -r/opt/stack/new/tempest/requirements.txt]; v = InvocationError('/opt/stack/new/tempest/.tox/tempest/bin/pip install -c/opt/stack/new/requirements/upper-constraints.txt -r/opt/stack/new/tempest/requirements.txt', 1)
2022-06-01 18:33:41.475 | + lib/tempest:install_tempest:1 : exit_trap

[1] https://zuul.opendev.org/t/openstack/build/09f7ff5b69b84e429e3141a622dfa951/log/controller/logs/grenade.sh_log.txt#16798
[2] https://zuul.opendev.org/t/openstack/build/0f98cf442e56467db4f67a5b3c6a6138/log/controller/logs/grenade.sh_log.txt#16769

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/805008
Committed: https://opendev.org/openstack/devstack/commit/e2bed1b72fcac3b689951372e4ac33784736c9aa
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit e2bed1b72fcac3b689951372e4ac33784736c9aa
Author: Ian Wienand <email address hidden>
Date: Wed Jul 28 11:19:57 2021 +1000

    Revert "Workaround for new pip 20.3 behavior"

    This reverts commit 7a3a7ce876a37376fe0dca7278e41a4f46867daa and
    bcd0acf6c0b5d6501e91133c3a937b3fc40f7122 and part of
    f1ed7c77c50ac28cb58c9f7ed885c6a3e0a75403 which all cap our pip
    installs.

    Given the pip ecosystem can often incorporate major changes, tracking
    upstream at least generally gives us one problem at a time to solve
    rather than trying to handle version jumps when LTS distros update.

    The new dependency resolver included some changes that disallow
    setting URL's like "file:///path/to/project#egg=project" in
    constraints. Apparently the fact it used to work was an accident of
    the requires/constraints mechanism; it does make some sense as the URL
    doesn't really have a version-number that the resolver can put in an
    ordering graph.

    The _setup_package_with_constraints_edit function comment highlights
    what this is trying to do

     # Updates the constraints from REQUIREMENTS_DIR to reflect the
     # future installed state of this package. This ensures when we
     # install this package we get the from source version.

    In other words; if constraints has "foo==1.2.3" and Zuul has checked
    out "foo" for testing, we have to make sure pip doesn't choose version
    1.2.3 from pypi.

    It seems like removing the entry from upper-requirements.txt is the
    important part; adding the URL path to the on-disk version was just
    something that seemed to work at the time, but isn't really necessary.
    We will install the package in question which will be the latest
    version (from Zuul checkout) and without the package in
    upper-requirements.txt nothing will try and downgrade it.

    Therefore the solution proposed here is to remove the adding of the
    URL parts.

    This allows us to uncap pip and restore testing with the new
    dependency resolver.

    Closes-Bug: #1906322
    Change-Id: Ib9ba52147199a9d6d0293182d5db50c4a567d677
    (cherry picked from commit 6b9a5646225a766f6240e2a1a93a92b82e088aa0)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/devstack/+/844774

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/devstack/+/844774
Reason: stable/victoria branch of openstack/devstack is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.