`--extra-index-url` not working for PIP

Bug #1833229 reported by Daniel Andersson on 2019-06-18
134
This bug affects 24 people
Affects Status Importance Assigned to Milestone
python-pip (Ubuntu)
High
James Page

Bug Description

I originally wrote this as a comment on
https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1822842 , but I
guess it makes sense to open a new separate bug since the other report was
already closed. Pasting my comment contents below.

    ~$ lsb_release -rd
    Description: Ubuntu 18.04.2 LTS
    Release: 18.04

    ~$ apt-cache policy python3-pip
    python3-pip:
      Installed: 9.0.1-2.3~ubuntu1.18.04.1
      Candidate: 9.0.1-2.3~ubuntu1.18.04.1
      Version table:
     *** 9.0.1-2.3~ubuntu1.19.04.1 500
            500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 packages
            500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe i386 packages
            100 /var/lib/dpkg/status
         9.0.1-2 500
            500 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 packages
            500 http://se.archive.ubuntu.com/ubuntu bionic/universe i386 packages

---

It seems like `9.0.1-2.3~ubuntu1.18.04.1` breaks the `--extra-index-url`
functionality of PIP.

In my understanding, the idea is for `--extra-index-url` to provide a
PyPI-compliant repository that offers a few additional packages, allowing PIP
to fall back onto the instance configured as `--index-url` (defaults to
upstream PyPI). This has been the case with earlier versions, allowing us to
host an internal PyPI repository containing a subset of packages, as well as a
generic local caching PyPI mirror.

In `9.0.1-2.3~ubuntu1.18.04.1`, it seems like the fallback mechanisms do not
work properly, instead failing if _either_ the `--index-url` _or_
`--extra-index-url` instances lack the package.

---

With `~/.config/pip/pip.conf` as:

    [global]
    index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple
    extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple

With `python3-pip` version `9.0.1-2.3~ubuntu1.18.04.1`, I get a session like:

    ~$ python3 -m venv env/pip-latest
    ~$ . env/pip-latest/bin/activate
    (pip-latest) ~$ pip --version
    pip 9.0.1 from /home/dandersson/env/pip-latest/lib/python3.6/site-packages (python 3.6)
    (pip-latest) ~$ pip install hpt
    Collecting hpt
    Exception:
    Traceback (most recent call last):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
        status = self.run(options, args)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run
        wb.build(autobuilding=True)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
        self.requirement_set.prepare_files(self.finder)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
        ignore_dependencies=self.ignore_dependencies))
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
        require_hashes
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
        self.link = finder.find_requirement(self, upgrade)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement
        all_candidates = self.find_all_candidates(req.name)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates
        for page in self._get_pages(url_locations, project_name):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages
        page = self._get_page(location)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page
        return HTMLPage.get_page(link, session=self.session)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page
        resp.raise_for_status()
      File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/

PIP fails with a traceback due to not finding `hpt` on the `--index-url`
instance -- but the point of giving `--extra-index-url` is that `hpt` resides
on that instance.

Trying to install a package that should be present on the `--index-url`
instance:

    (pip-latest) ~$ pip install requests
    Collecting requests
    Exception:
    Traceback (most recent call last):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
        status = self.run(options, args)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run
        wb.build(autobuilding=True)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
        self.requirement_set.prepare_files(self.finder)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
        ignore_dependencies=self.ignore_dependencies))
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
        require_hashes
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
        self.link = finder.find_requirement(self, upgrade)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement
        all_candidates = self.find_all_candidates(req.name)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates
        for page in self._get_pages(url_locations, project_name):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages
        page = self._get_page(location)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page
        return HTMLPage.get_page(link, session=self.session)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page
        resp.raise_for_status()
      File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_extra/simple/requests/

Now it fails with a traceback since `requests` does not exist on the
`--extra-index-url` instance, but the purpose of that instance is to provide a
small subset of extra packages, not to be a full mirror.

Changing `~/.config/pip/pip.conf` to:

    [global]
    index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple

removing the `--extra-index-url` instance altogether, it behaves as expected,
not finding the `hpt` package, but being able to install `requests`:

    (pip-latest) ~$ pip install hpt
    Collecting hpt
    Exception:
    Traceback (most recent call last):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
        status = self.run(options, args)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run
        wb.build(autobuilding=True)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
        self.requirement_set.prepare_files(self.finder)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
        ignore_dependencies=self.ignore_dependencies))
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
        require_hashes
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
        self.link = finder.find_requirement(self, upgrade)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement
        all_candidates = self.find_all_candidates(req.name)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates
        for page in self._get_pages(url_locations, project_name):
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages
        page = self._get_page(location)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page
        return HTMLPage.get_page(link, session=self.session)
      File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page
        resp.raise_for_status()
      File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/

It is expected to not be able to install `hpt`, but PIP should not fail with a
traceback, but just with a diagnostic message like `No matching distribution
found for hpt`.

Installing a common package will correctly use the `--index-url` instance:

    (pip-latest) ~$ pip install requests
    Collecting requests
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
    Collecting idna<2.9,>=2.5 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
    Collecting certifi>=2017.4.17 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl
    Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
    Collecting chardet<3.1.0,>=3.0.2 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
    Installing collected packages: idna, certifi, urllib3, chardet, requests
    Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3

---

Downgrading `python3-pip` to `9.0.1-2`:

    ~$ sudo apt install python3-pip=9.0.1-2 python-pip-whl=9.0.1-2
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following packages will be DOWNGRADED:
      python-pip-whl python3-pip
    0 upgraded, 0 newly installed, 2 downgraded, 0 to remove and 5 not upgraded.
    Need to get 1 493 kB of archives.
    After this operation, 226 kB disk space will be freed.
    Do you want to continue? [Y/n]
    Get:1 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pip all 9.0.1-2 [114 kB]
    Get:2 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python-pip-whl all 9.0.1-2 [1 379 kB]
    Fetched 1 493 kB in 0s (4 177 kB/s)
    dpkg: warning: downgrading python3-pip from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2
    (Reading database ... 196581 files and directories currently installed.)
    Preparing to unpack .../python3-pip_9.0.1-2_all.deb ...
    Unpacking python3-pip (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ...
    dpkg: warning: downgrading python-pip-whl from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2
    Preparing to unpack .../python-pip-whl_9.0.1-2_all.deb ...
    Unpacking python-pip-whl (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ...
    Setting up python-pip-whl (9.0.1-2) ...
    Setting up python3-pip (9.0.1-2) ...
    Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

and retrying the first attempt with `~/.config/pip/pip.conf` as:

    [global]
    index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple
    extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple

yields a session like:

    ~$ python3 -m venv env/pip-previous
    ~$ . env/pip-previous/bin/activate
    (pip-previous) ~$ pip --version
    pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6)
    (pip-previous) ~$ pip install hpt
    Collecting hpt
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/hpt/0.1.397/hpt-0.1.397.tar.gz
    Collecting argh (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/06/1c/e667a7126f0b84aaa1c56844337bf0ac12445d1beb9c8a6199a7314944bf/argh-0.26.2-py2.py3-none-any.whl
    Collecting pyyaml==5.1 (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/9f/2c/9417b5c774792634834e730932745bc09a7d36754ca00acf1ccd1ac2594d/PyYAML-5.1.tar.gz
    Collecting argcomplete (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/4d/82/f44c9661e479207348a979b1f6f063625d11dc4ca6256af053719bbb0124/argcomplete-1.10.0-py2.py3-none-any.whl
    Collecting distro (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/ea/35/82f79b92fa4d937146c660a6482cee4f3dfa1f97ff3d2a6f3ecba33e712e/distro-1.4.0-py2.py3-none-any.whl
    Collecting requests (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
    Collecting termcolor (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
    Collecting cilib==v0.1.397 (from hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/cilib/0.1.397/cilib-0.1.397.tar.gz
    Collecting idna<2.9,>=2.5 (from requests->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
    Collecting chardet<3.1.0,>=3.0.2 (from requests->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
    Collecting certifi>=2017.4.17 (from requests->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl
    Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
    Collecting hvac (from cilib==v0.1.397->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/7b/2c/b95b551769459406ca440788c32d699820f4c875020ff95c6e3844eee6d3/hvac-0.9.2-py2.py3-none-any.whl
    Collecting psycopg2-binary (from cilib==v0.1.397->hpt)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/b1/35/75c9c2d9cfc073ab6c42b2d8e91ff58c9b99f4ed7ed56b36647642e6080e/psycopg2_binary-2.8.3-cp36-cp36m-manylinux1_x86_64.whl
    [...SNIP...]
    Successfully installed argcomplete-1.10.0 argh-0.26.2 certifi-2019.6.16 chardet-3.0.4 cilib-0.1.397 distro-1.4.0 hpt-0.1.397 hvac-0.9.2 idna-2.8 psycopg2-binary-2.8.3 pyyaml-5.1 requests-2.22.0 termcolor-1.1.0 urllib3-1.25.3

PIP now correctly resolves the respective packages to their respective
instances, using `--extra-index-url` for the packages that are _only_ present
there, and falling back to `--index-url` for everything else.

Changing `~/.config/pip/pip.conf` to:

    [global]
    index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple

cleaning out the virtual environment and trying again:

    (pip-previous) ~$ deactivate
    ~$ rm -rf env/pip-previous/
    ~$ python3 -m venv env/pip-previous
    ~$ . env/pip-previous/bin/activate
    (pip-previous) ~$ pip --version
    pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6)
    (pip-previous) ~$ pip install hpt
    Collecting hpt
      Could not find a version that satisfies the requirement hpt (from versions: )
    No matching distribution found for hpt
    (pip-previous) ~$ pip install requests
    Collecting requests
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
    Collecting chardet<3.1.0,>=3.0.2 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
    Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
    Collecting certifi>=2017.4.17 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl
    Collecting idna<2.9,>=2.5 (from requests)
      Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
    Installing collected packages: chardet, urllib3, certifi, idna, requests
    Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3

---

In summary, from my point-of-view, the `9.0.1-2.3~ubuntu1.18.04.1` version
breaks all functionality related to `--extra-index-url`, as well as regresses
the behaviour when a package is not found when using `--index-url`.

I will be happy to provide any additional information that can help.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-pip (Ubuntu):
status: New → Confirmed
information type: Public → Public Security
information type: Public Security → Private Security
information type: Private Security → Public
Valtteri Mäkelä (vltrrr) wrote :

Confirmed also with Mint 19.1 with latest packages, if using our internal pypi server. Creating a plain py3 virtualenv fails:

$ virtualenv testenv --python python3

Fails with:

  File "/home/user/testenv/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.internal.com/simple/setuptools/

Disabling the internal pypi server from ~/.pip/pip.conf works, but of course removes access to internal packages.

Downgrading to previous version as suggested by Daniel Anderson resolved the issue:

sudo apt install python3-pip=9.0.1-2 python-pip-whl=9.0.1-2

Marc Mercer (daemoen) wrote :

Can confirm here as well. I also tested the pip/whl builds from universal which are at 18.x -- these unfortunately have the exact same issue. Was hoping maybe they had moved past it, which suggests that the issue is with the unvendored requests, as opposed to pip itself. https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1842120 can be marked as a duplicate of this.

tags: added: regression-update
Johnny (jairoalonso25) wrote :

Can Confirm. Using a private PyPI repo in my company.

pip install --extra-index-url=https://pypi.pipelineservices.alertlogic.com alps
Collecting alps
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/alps/
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/alps/

lsb_release -rd
Description: Ubuntu 18.04.3 LTS
Release: 18.04

Using WLS form Windows 10

James Page (james-page) wrote :

I'm slightly baffled by this as the original bug report and associated fix resolved an issue where:

  requests.exceptions.HTTPError

was not the same as requests.HTTPError in the distro packaging - but the exception raised in this bug report infers that something is still wonky in the debundling of the vendored requests...

James Page (james-page) wrote :

Struggling to reproduce this issue; I've setup a test instance with a extra-index-url which points to an empty index; this throws a 404 when accessed for the modules being requested but the exception handler is correctly catching the exception, falling back to the main package index.

James Page (james-page) wrote :

404 Client Error: Not Found for url: http://mirror.dfw.rax.openstack.org/wheel/urllib3/
Traceback (most recent call last):
  File "/root/pip-latest/lib/python3.6/site-packages/pip/index.py", line 796, in get_page
    resp.raise_for_status()
  File "/root/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
pip._vendor.requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://mirror.dfw.rax.openstack.org/wheel/urllib3/

As you can see the correct/vendor path-ed exception is being raised and caught.

James Page (james-page) on 2019-09-24
Changed in python-pip (Ubuntu):
status: Confirmed → Incomplete
Marc Mercer (daemoen) wrote :

@james-page I think the thing being overlooked is what it is doing, not wether or not the exception is correct. In our case, we have repositories that have some artifacts, and repositories that may have other artifacts... (and once in a rare while, overlap with different versions).. If we request a package that does *not* exist in the custom urls, then it will fail completely, instead of passing with the only failure being on that one repository. By default, it should check each of the repositories, see if they have it, etc.. It seems to be checking them all, but if *any* of them fails to have an artifact being requested, the entire pip operation implodes, even if the package *is* available in a different repository successfully prior to the fetching of the failed repo.

James Page (james-page) wrote :

The exception drives the behaviour so the behaviour and the incorrect exception class are linked - the pip index.py code is mismatching on the type HTTPError resulting in an unhandled exception and the behaviour you are seeing.

I can't reproduce this at the moment so there must be something we're missing.

James Page (james-page) wrote :

https://github.com/pypa/pip/commit/e4936da2e1b62c33f0ed9034f5c984ae17e38e86 might be related - I'll prepare a test package to see if that drives the correct behaviour in all cases.

James Page (james-page) wrote :

I believe this is something todo with missing vendor pathing for requests.exceptions.

I've picked the commit in #10 into the bionic package and uploaded to:

  https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3690

as I'm unable to reproduce the issue it would be helpful if one of the impacted bug participants could verify it resolves the issue.

James Page (james-page) on 2019-09-25
Changed in python-pip (Ubuntu):
status: Incomplete → New
importance: Undecided → High
James Page (james-page) on 2019-09-26
Changed in python-pip (Ubuntu):
status: New → Incomplete
assignee: nobody → James Page (james-page)
Marc Mercer (daemoen) wrote :

@james-page Tested -- not fixed

root@vagrant:/etc# dpkg -l | grep pip
ii libpipeline1:amd64 1.5.0-1 amd64 pipeline manipulation library
ii python-pip 9.0.1-2.3~ubuntu1.18.04.2~ubuntu18.04.1~ppa201909251252 all Python package installer
ii python-pip-whl 9.0.1-2.3~ubuntu1.18.04.2~ubuntu18.04.1~ppa201909251252 all Python package installer
ii python3-pip 9.0.1-2.3~ubuntu1.18.04.2~ubuntu18.04.1~ppa201909251252 all Python package installer
root@vagrant:/etc# pip install ansible==2.8.0
Collecting ansible==2.8.0
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 404 Client Error: Not Found for url: https://artifacts.ourdomain.com/repository/pypi-release/simple/ansible/

Marc Mercer (daemoen) wrote :

@james-page -- Please ping me on freenode (Daemoen) on the ubuntu channel. I have created a user for you to test against our repository host, as hopefully that should simplify this a bit. I can provide you all of the details for the configuration needed, but don't want to provide them here as this is a public medium. I just tested the credentials and ensured that it will allow you to replicate the issue as the rest of us see it.

chbrosso (chbrosso) wrote :

It looks like the upstream issue is here, and it mentions that it's a vendoring issue:

https://github.com/pypa/pip/issues/6799

Akkana Peck (akkzilla) wrote :

This might be a simpler way to reproduce it. I uploaded a package to test.pypi and wanted to use --extra-index-url to verify that its dependencies are pulled in (it has dependencies that are in pypi but not in test.pypi). Here are my steps to reproduce it:

python3 -m venv /tmp/test3env
source /tmp/test3env/bin/activate
pip install wheel
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pytopo==1.6.1

which gives the error:

Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://test.pypi.org/simple/pycairo/

Sometimes it errors on a different package, not always pycairo, but the rest of the error is the same. I filed https://github.com/pypa/pip/issues/7486 upstream and I suspect they're right that it's the same issue as this Ubuntu bug (I'm on 19.10).

chbrosso (chbrosso) on 2020-01-14
Changed in python-pip (Ubuntu):
status: Incomplete → Confirmed
chbrosso (chbrosso) wrote :

A simple command to test for the issue is pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple simplejson==3.17

@james-page I tested your proposed PPA with this command, and the issue is still here.

I gave a shot to this, and generated a patch from the upstream pull request https://github.com/pypa/pip/pull/6367. I could test it by getting source package, patching it, and building it back, I can confirm that the issue is fixed this way.

I'm unable to further propose a new package as I'm not used to contributing fixes, but would be happy to if necessary.

The attachment "Fix-1833229.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
chbrosso (chbrosso) wrote :

@james-page (or any other maintainer), did you have a chance to look at the patch? I'm pretty confident it solves the issue.

James Page (james-page) wrote :

sorry - I missed this update - looking now!

James Page (james-page) wrote :

pip right through to Ubuntu development appears to be missing part of the fix for this problem so I've picked fixes for bionic/eoan/focal into:

  https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3690

I'm still not able to reproduce this issue but please test and see if the proposed version resolves this issue with the test case in #16.

James Page (james-page) wrote :

Right - I was able to reproduce this issue on focal using:

python3 -m venv /tmp/test3env
source /tmp/test3env/bin/activate
pip install wheel
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pytopo==1.6.1

(thanks for that)

However with the patched package in the PPA, I now hit a completely different problem with the test case which looks related to the patch:

Traceback (most recent call last):
  File "/usr/lib/python3.8/ensurepip/__init__.py", line 54, in _run_pip
    import pip._internal
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpzye4xja3/pip-18.1-py2.py3-none-any.whl/pip/_internal/__init__.py", line 40, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpzye4xja3/pip-18.1-py2.py3-none-any.whl/pip/_internal/cli/autocompletion.py", line 8, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpzye4xja3/pip-18.1-py2.py3-none-any.whl/pip/_internal/cli/main_parser.py", line 8, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpzye4xja3/pip-18.1-py2.py3-none-any.whl/pip/_internal/cli/cmdoptions.py", line 22, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpzye4xja3/pip-18.1-py2.py3-none-any.whl/pip/_internal/utils/ui.py", line 15, in <module>
ModuleNotFoundError: No module named 'pip._vendor.progress.helpers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.8/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/usr/lib/python3.8/ensurepip/__init__.py", line 263, in _main
    return _bootstrap(
  File "/usr/lib/python3.8/ensurepip/__init__.py", line 181, in _bootstrap
    return _run_pip(args + _PROJECTS, additional_paths)
  File "/usr/lib/python3.8/ensurepip/__init__.py", line 58, in _run_pip
    return pip.main(args)
AttributeError: module 'pip' has no attribute 'main'

unknown (charstar) wrote :

Applying https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3690 fixed the issues I was having.

Before:

$ pip3 install --trusted-host packages.[redacted].com --extra-index-url http://packages.[redacted].com/software/python --extra-index-url https://[redacted].com/pypi/simple/ talos-mrt
Collecting talos-mrt
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/talos-mrt/

After:

$ pip3 install --trusted-host packages.[redacted].com --extra-index-url http://packages.[redacted].com/software/python --extra-index-url https://[redacted].com/pypi/simple/ talos-mrt
Collecting talos-mrt
  Downloading http://packages.[redacted].com/software/python/talos-mrt/talos-mrt-1.0.1.tar.gz (40kB)
...
Building wheels for collected packages: talos-mrt, ...
...
Successfully built talos-mrt ...

Thanks!

Gerard Weatherby (gweatherby) wrote :

To make this copy-and-paste obvious for the next person to come by:

add-apt-repository ppa:ci-train-ppa-service/3690
apt update
apt install python3-pip

Akkana Peck (akkzilla) wrote :

When I use the python3-pip from ppa:ci-train-ppa-service/3690 and run:
/usr/bin/pip3 install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple metapho
in a virtualenv that's brand new except that I've installed wheel, I get:

Looking in indexes: https://test.pypi.org/simple/, https://pypi.org/simple
Requirement already satisfied: metapho in /home/akkana/.local/lib/python3.7/site-packages (1.0)
Requirement already satisfied: pycairo in /usr/lib/python3/dist-packages (from metapho) (1.16.2)
Requirement already satisfied: PyGObject in /usr/lib/python3/dist-packages (from metapho) (3.34.0)

Is that what it's supposed to do? I created the virtualenv with python3 -m venv /tmp/testpythonenv, no --system-site-packages, and what I want it to do is pull the requirements from pypi.org, not find them in /usr/lib/python3. (It also shouldn't be putting anything in .local when I'm using a venv, but that's a different issue.)

Does anyone know if this is thought to be fixed in Fossa?

Scott Kitterman (kitterman) wrote :

The fix was included in pip 20.0.2 brought in from Debian:

 python-pip | 20.0.2-5ubuntu1 | focal/universe | source
 python-pip | 20.0.2-5ubuntu1 | groovy/universe | source

Changed in python-pip (Ubuntu):
status: Confirmed → Fix Released
andrei (akela1101) wrote :

I'm not sure how the LTS system works, but for me this doesn't look Fixed if bug is still present in bionic.

I guess, a quick workaround is to put pip2 >=20.0.2 whl to the extra-index-url repo and use it.

Gerard Weatherby (gweatherby) wrote :

It appears the ppa is no longer "just working" because the mainline bionic Ubuntu version of python3-pip has incremented. That is to say, even after adding the ppa and doing apt update, apt-get install pulls the buggy one from the mainline repo.

I was able to get the right version by using:
apt install python3-pip=9.0.1-2.3~ubuntu1.18.04.2~ubuntu18.04.1~ppa202002141134 python-pip-whl=9.0.1-2.3~ubuntu1.18.04.2~ubuntu18.04.1~ppa202002141134

Gerard Weatherby (gweatherby) wrote :

To stop automatic updates from overwriting the bug fix version of python3-pip, create file
/etc/apt/preferences.d/citrain-pin-700 with the following content:
Package: python3-pip
Pin: origin "ppa.launchpad.net"
Pin-Priority: 700

Gerard Weatherby (gweatherby) wrote :

Yesterday's (10-13) update of the PPA has broken the fix.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.