venv creation fails: ensurepip error

[Jibben Nee]

Tested on focal with both the focal-proposed 3.9.0-1~20.04 amd64 and focal-updates 3.9.0~rc1-1~20.04 amd64 packages.

Installed python3.9 (+deps) and python3.9-venv

$ python3.9 -m venv venv
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/home/ziddey/test/venv/bin/python3.9', '-Im', 'ensurepip', '--upgrade', '--default-pip']

$ python3.9 -m ensurepip --version
/usr/bin/python3.9: Error while finding module specification for 'ensurepip.__main__' (ImportError: cannot import name '_bundled' from partially initialized module 'ensurepip' (most likely due to a circular import) (/usr/lib/python3.9/ensurepip/__init__.py))

vs:

$ python3.8 -m ensurepip --version
pip 20.0.2

Adding --without-pip works to create the venv, but obviously without pip.

Comparing /usr/lib/python3.9/ensurepip/__init__.py with the official cpython, it looks like we don't bundle. Instead, we use common python wheels in /usr/share/python-wheels/. This means that python3.9 would end up sharing the same wheels as python3.8.

By commenting out:

from . import _bundled

everything works fine. venv creates/activates using the common wheels made available from python 3.8 (unless the same old versions are also being bundled with 3.9?) to create the venv with --upgrade-deps so that it'll take on current versions instead.

Of course, this relies on having the wheels available (didn't check if 3.9 also includes these wheels or if they're pre-existing from 3.8) and having them be compatible (they target py2.py3 so this should be fine).

$ python3.9 -m venv --upgrade-deps venv
Collecting pip
  Downloading pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 2.8 MB/s
Collecting setuptools
  Downloading setuptools-50.3.1-py3-none-any.whl (785 kB)
     |████████████████████████████████| 785 kB 17.3 MB/s
Installing collected packages: pip, setuptools
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
  Attempting uninstall: setuptools
    Found existing installation: setuptools 44.0.0
    Uninstalling setuptools-44.0.0:
      Successfully uninstalled setuptools-44.0.0
Successfully installed pip-20.2.3 setuptools-50.3.1
ziddey@laurie:~/test$ . venv/bin/activate
(venv) ziddey@laurie:~/test$ pip list
Package Version
------------- -------
pip 20.2.3
pkg-resources 0.0.0
setuptools 50.3.1

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jibben Nee (ziddey)]

Just saw this has been fixed upstream: https://salsa.debian.org/cpython-team/python3/-/commit/5bd846b0e8096f9729e216be6f92450a90a9711e

Thanks Matthias Klose