14.04LTS upgrades to pip cause object has no attribute 'findall'

[Bryce Nesbitt]

# pip install --upgrade six
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1096, in prepare_files
    req_to_install, self.upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 256, in find_requirement
    page_versions.extend(self._package_versions(page.links, req.name.lower()))
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 432, in _package_versions
    for link in self._sort_links(links):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 422, in _sort_links
    for link in links:
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 769, in links
    for anchor in self.parsed.findall(".//a"):
AttributeError: 'Document' object has no attribute 'findall'

Storing debug log for failure in /root/.pip/pip.log

------------------------
See
https://github.com/pypa/pip/issues/1742
for the upstream, which blames the distro

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: python-pip 1.5.4-1
ProcVersionSignature: Ubuntu 3.13.0-35.62-generic 3.13.11.6
Uname: Linux 3.13.0-35-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.14.1-0ubuntu3.4
Architecture: amd64
Date: Tue Sep 23 23:46:17 2014
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_US
 TERM=xterm
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: python-pip
UpgradeStatus: Upgraded to trusty on 2014-09-18 (6 days ago)

[Bryce Nesbitt (bryce2)]

# sudo pip install apscheduler
Downloading/unpacking apscheduler
  Downloading APScheduler-3.0.0-py2.py3-none-any.whl (46kB): 46kB downloaded
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1259, in prepare_files
    )[0]
IndexError: list index out of range

[Bryce Nesbitt (bryce2)]

Note this affects ANY tool that uses pip or virtualenv. I encountered it first with PyCharm,
then later when installing a commercial backup software (iDrive Linux Desktop).

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[launchpadmember (lpuser1138)]

From the forum for the package team responsible for PIP comes the following comments on this issue:
-------- 1st comment --------
dstufft commented on May 26, 2014 <Python Packaging Authority member>

Ah I see.

So pip itself bundles some software, html5lib being one of those, on various Linux distros they unbundle that software and cause pip to import it normally instead of using it's own bundled copy.

What this means is that if you install an incompatible version of one of pip's dependencies into the global Python, then you'll break pip because pip will import that and attempt to use that instead of the correct one installed by apt-get.

-------- 2nd comment (a clarification of the 1st comment above) --------
Ivoz commented on Jun 4, 2014 < Python Packaging Authority member>

@lrossouw try pip installing packages in a virtualenv or using the --user flag instead of sudo installing them as system packages.

The issue seems to be caused by debian's modification of its pip, so we can't address it ourselves.

-------- Source for above comments: --------
https://github.com/pypa/pip/issues/1742

I'm including these comments as a record that the dev's in charge of PIP suggested that this issue belongs here. This may not be necessary but it is a record of the events that led here and as such I'm hoping it is at least informative if not helpful to some people.

[Kai Kasurinen (kai-kasurinen)]

Since it seems likely to me that this is a local configuration problem, rather than a bug in Ubuntu, I am marking this bug as 'Incomplete'.

However, if you believe that this is really a bug in Ubuntu, then we would be grateful if you would provide a more complete description of the problem with steps to reproduce, explain why you believe this is a bug in Ubuntu rather than a problem specific to your system, and then change the bug status back to "New".

For local configuration issues, you can find assistance here: http://www.ubuntu.com/support/community

[Paul White (paulw2u)]

This issue has sat incomplete for more than 60 days now without any response so I'm closing it now as "Invalid".

Please feel free to re-open this bug report if this is still an issue when using any currently supported release of Ubuntu.