sudo pip install installs into the system dist-packages by default

[Matthias Klose]

sudo pip install installs into the system dist-packages by default. We didn't rename site-packages to dist-packages, and have the next random upstream tool mess up with the system python. Needs to be resolved as soon as possible

[Zach Heilman (organman91)]

Not sure if this is exactly the same bug, but I'm seeing pip break itself. Packages are being installed in /usr/local/lib/python2.7/dist-packages/ and at some point it's installing the requests package. This breaks pip completely:

Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.4', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 351, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2363, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2088, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 11, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar # noqa
  File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 23, in <module>
    from requests.adapters import BaseAdapter
ImportError: No module named adapters

Moving /usr/local/lib/python2.7/dist-packages out of the way fixes the problem.

[Zach Heilman (organman91)]

Whoops, sorry, this isn't the same bug. But "sudo pip install requests" breaks pip.

[Zach Heilman (organman91)]

Well I'd like to apologize for that whole faff. Apparently the version of requests specified in our requrirements.txt was 0.14.1 [!] - please disregard above comments.

[Michael Vogt (mvo)]

@doko: I just tried to reproduce this but for me it installs into /usr/local/lib/python2.7/dist-packages by default. If I manually set the prefix to "/usr" instead of /usr/local it goes to /usr/lib/python2.7/site-packages (as expected). The root is the install schema that is selected by distutils. Thats in the first case unix_local and in the later unix_prefix.

[Michael Vogt (mvo)]

We might consider changing the default target for unix_local to site-packages as well. Or is there a reason that we keep it like this?

[Michael Vogt (mvo)]

Fwiw, I can not reproduce this with pyhon2,3 with or without virtualenv. "sudo pip install enum34" (as a example) installs into /usr/local/lib/python2.7/dist-package, but not in the system /usr/lib/python2.7/dist-packages dir. Same for sudo pip3 install enum34 (/usr/local/lib/python3.4/dist-packages).

Help how to reproduce this would be appreciated.

[Barry Warsaw (barry)]

Michael is exactly right. `sudo pip install` (and pip3) will install into /usr/local/lib/pythonX.Y/dist-packages by default (no switches), which is where they *should* get installed. I'm not sure there's a still valid bug here.

[Barry Warsaw (barry)]

After talking with Matthias, I think I understand the gist of this bug.

Let's say I apt-get install python3.4 (well, a modified package that re-enables the ensurepip module). Now I use pyvenv-3.4 to create a virtual environment. With ensurepip enabled, <venv>/bin will contain pip, pip3, and pip3.4 executables. If you enter the venv (by sourcing bin/activate), those will do the right thing by installing into the venv.

However, let's say you move those aside and apt-get install python3-pip. Now enter the venv and run pip3 install enum34. That package will get installed into /usr/lib/python3.4/site-packages! Not dist-packages, and not /usr/local.

That seems bad, but I'm not sure how bad it will be in practice. If we re-enable ensurepip as in LP: #1290847 then first off, anybody entering the venv will use the <venv>/bin/pip script instead and that will work as expected. You'd have to actively manipulate your venv in a way that I don't think makes much sense, or if you do it, you know what your doing (or think you do :). If you don't run system pip3 as root, you just get the expected permission denied.

There's still the possibility that you enter the venv, delete <venv>/bin/pip*, apt-get install python3-pip, and sudo pip3 install something. Yes, that will lead to broken stuff and you pretty much had to willfully want to break your system, but if that's really a problem, then python3-pip could be changed to refuse to run if sys.prefix != sys.base_prefix (i.e. you are in a virtual environment). That would force you to use the ensurepip installed pip* executables. But of course then we'd have to resolve the problems with ensurepip.

[Marc (marc2014)]

This bug matters. I have a pretty vanilla installation, python, matplotlib, and a few other standard packages, all from the official repositories. After a recent apt-get update, I now get the same exact pip breaking result as the first comment.

The workaround was to use easy_install to install pip, then use pip2.7 to install what I wanted. But I presume that is not the intended path for typical users.

[CSRedRat (csredrat)]

I don't understand what's happening with unfinished tasks in the transition to the development of new release. And my concern about counter "There are 100 days left in the current cycle. That means that in order to complete all the work 2.68 workitems must be completed per day.". Toward the end of the developing process it is ~100-200 workitems must be completed per day and not completed.

Many "critical" bugs on ReleaseNotes Trusty Tahr page don't fixed presently: https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes#Known_issues
Installation bugs too:
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1066480
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1172572
https://bugs.launchpad.net/ubuntu/+source/graphite2/+bug/1303516
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1297851
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1066342
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1172161
https://bugs.launchpad.net/ubuntu/+source/console-setup/+bug/1297234

upgrade:
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1308530
https://bugs.launchpad.net/ubuntu/+source/tex-common/+bug/1304972
https://bugs.launchpad.net/ubuntu/+source/flightgear/+bug/1308338

and other:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1054732
https://bugs.launchpad.net/unity/+bug/1305586
https://bugs.launchpad.net/ubuntu/+source/gnome-keyring/+bug/1271591
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1308037
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1305522
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1308761
https://bugs.launchpad.net/ubuntu/+source/unity-greeter/+bug/1292467

Too many errors in Trusty on: http://errors.ubuntu.com
Please fix the most annoying bugs. It interferes with work, use and distribution advice to friends.

[Eero Nevalainen (enevalainen)]

Now this is a pretty nasty bug which I wouldn't expect to see on a very standard Ubuntu installation. Currently this is a big impediment for me at work as I need to get the robot framework working but instead I find myself fighting some fundamental bug in how Python packages get installed...

[j00bar (jag)]

More information on https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1694560