python3 -m venv installs pip to incorrect path VENV_ROOT/local/bin/pip

[Anders Kaseorg]

python3 -m venv worked correctly in Ubuntu 21.10:

# apt update; apt install python3.10-venv
# python3.10 -m venv /tmp/my-venv
# . /tmp/my-venv/bin/activate
# type pip
pip is /tmp/my-venv/bin/pip
# pip --version
pip 20.3.4 from /tmp/my-venv/lib/python3.10/site-packages/pip (python 3.10)

But in Ubuntu 22.04, it installs pip to the wrong path:

# apt update; apt install python3.10-venv
# python3.10 -m venv /tmp/my-venv
# . /tmp/my-venv/bin/activate
# type pip
bash: type: pip: not found
# pip --version
bash: pip: command not found
# echo $PATH
/tmp/my-venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# ls /tmp/my-venv/bin
Activate.ps1 activate.csh python python3.10
activate activate.fish python3
# ls /tmp/my-venv/local/bin
pip pip3 pip3.10

[Anders Kaseorg (andersk)]

Trying virtualenv instead of venv, it installs pip to the right path, but if you use it to install other packages before activating the virtualenv, their binaries end up in the wrong path. Ubuntu 21.10:

# apt update; apt install python3-virtualenv
# virtualenv /tmp/my-virtualenv
# /tmp/my-virtualenv/bin/pip install black
# . /tmp/my-virtualenv/bin/activate
# type black
black is /tmp/my-virtualenv/bin/black
# black --version
black, 22.1.0 (compiled: yes)

Ubuntu 22.04:

# apt update; apt install python3-virtualenv
# virtualenv /tmp/my-virtualenv
# /tmp/my-virtualenv/bin/pip install black
# . /tmp/my-virtualenv/bin/activate
# type black
bash: type: black: not found
# black --version
bash: black: command not found
# ls /tmp/my-virtualenv/local/bin
black black-primer blackd

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dan Halbert (dhalbert)]

Confirmed: I ran into this independently. I tried uninstalling python3.9-venv, just in case it was interfering, and updating pip with itself to 22.0.3, and neither helped.

$ python3.10 -mvenv 310
$ python3.9 -mvenv 39
$ ls 310/bin
activate activate.csh activate.fish Activate.ps1 python python3 python3.10
$ ls 39/bin
activate activate.csh activate.fish Activate.ps1 pip pip3 pip3.10 pip3.9 python python3 python3.9

[Anders Kaseorg (andersk)]

sysconfig-debian-schemes.patch uses an incorrect test for a virtual environment:

if 'real_prefix' in sys.__dict__ or 'VIRTUAL_ENV' in os.environ:

Both sys.real_prefix and VIRTUAL_ENV are only set by virtualenv, not venv, and then only when the virtualenv is activated by VENV_ROOT/bin/activate, not when VENV_ROOT/bin/python is launched directly by its path.

This patch corrects the test to

if sys.base_prefix != sys.prefix:

which works on Python ≥ 3.3.

(Also submitted to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006707.)

[Dan Halbert (dhalbert)]

For anyone looking for a workaround, thanks to andersk above. I edited
/usr/lib/python3.10/sysconfig.py per this diff:

usr/lib/python3.10$ diff -p sysconfig.py~ sysconfig.py
*** sysconfig.py~ 2022-02-26 03:21:15.000000000 -0500
--- sysconfig.py 2022-03-10 21:09:02.572726493 -0500
*************** def _get_preferred_schemes():
*** 262,268 ****
              'user': 'osx_framework_user',
          }

! if 'real_prefix' in sys.__dict__ or 'VIRTUAL_ENV' in os.environ:
          # virtual environments
          prefix_scheme = 'posix_prefix'
      else:
--- 262,270 ----
              'user': 'osx_framework_user',
          }

! # https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1962791/comments/4
! if sys.base_prefix != sys.prefix:
! #if 'real_prefix' in sys.__dict__ or 'VIRTUAL_ENV' in os.environ:
          # virtual environments
          prefix_scheme = 'posix_prefix'
      else:

[Stefano Rivera (stefanor)]

FYI, I filed https://salsa.debian.org/cpython-team/python3/-/merge_requests/18 including a fix for this.

[Anders Kaseorg (andersk)]

Fixed in 3.10.2-7 in jammy-proposed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python3.10 - 3.10.2-7

---------------
python3.10 (3.10.2-7) unstable; urgency=medium

  [ Stefano Rivera ]
  * Use the same is_virtual_environment() proposed in PEP 668 across the site,
    sysconfig, and distutils modules. Stop checking for PYTHONUSERBASE or
    VIRTUAL_ENV environment variables.
  * Correct assertion descriptions in module-install-* autopkgtests.
  * Extend module-install-* autopkgtests to catch the issues fixed in -6:
    - Check the directory that the modules are installed to.
    - Check editable installs.
    - Check pip-driven PEP-517 installs.
    - Check that virtualenvs are seeded with the correct layout.
  * Re-enable module-install-virtualenv.

 -- Matthias Klose <email address hidden> Mon, 14 Mar 2022 06:18:40 +0100