editable mode crash

[Barry Warsaw]

[Impact]

From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=823358

pip install -e .[testing]

crashes.

The `[testing]` bit names an `extras_require` section in a package's setup.py so this crash prevents the installation of package extras.

The coming SRU fixes to python-pip backports into a quilt patch just enough of the upstream changes to repair installing extras.

This does however require an SRU for pyparsing because it is a new runtime dependency of the patch, and the version of pyparsing in Xenial cannot be dirtbiked (rewheeled) because it uses distutils.setup() instead of setuptools.setup(). So the SRU for pyparsing simply patches the setup.py to use setuptools and thus produce an egg-info directory instead (which contains the necessary metadata for dirtbike to turn this new Built-Using package into a .whl).

Neither fix is necessary for Yakkety because 1) pip has a new upstream release in Debian which contains the backported fix; 2) pyparsing has a new upstream release in Debian which already switches to setuptools.

Both should show up in Yakkety once the normal unstable autoimport runs.

[Test Case]

I've also backported a DEP-8 test called pip3-editable.sh which proves that this change fixes the bug. I added this DEP-8 test in the new Debian version, and proved that running the DEP-8 tests without the quilt patch fails, but with it, it succeeds.

[Regression potential]

It should be low to none. The previously existing DEP-8 tests invoke pip under several common scenarios (i.e. both inside and outside a virtualenv, and both as a normal user and root).

All tests continue to pass, although I'll note that I also had to backport the addition of PIP_DISABLE_PIP_VERSION_CHECK=1 from the new Debian package to prevent stderr output due to the new upstream version.

[Barry Warsaw (barry)]

Debian bug has the details, but this does affect Xenial so will need an SRU

[Barry Warsaw (barry)]

pyparsing is a new dependency of the cherry pick for pip so it needs to be dirtbiked too. However, pyparsing is using distutils.setup() and that won't create the egg-info directory needed to produce the necessary metadata. So pyparsing will need an SRU fix too.

[Chris Halse Rogers (raof)]

Hello Barry, or anyone else affected,

Accepted pyparsing into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pyparsing/2.0.3+dfsg1-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Barry, or anyone else affected,

Accepted python-pip into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-pip/8.1.1-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Barry Warsaw (barry)]

The SRU package needed a rebuild, but now that that's done and it got published to xenial-proposed, I've verified that the originally failing pip command works now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-pip - 8.1.1-2ubuntu0.1

---------------
python-pip (8.1.1-2ubuntu0.1) xenial; urgency=medium

  * d/patches/editable-mode.patch: Cherry pick upstream fixes for crash
    when pip installing in editable mode. (LP: #1578761)
  * d/tests:
    - pip3-editable.sh: Added to prove the crash is fixed.
    - control: Updated
    - *.sh: export PIP_DISABLE_PIP_VERSION_CHECK=1 to prevent stderr
      warnings (and thus test failures) because of the newer upstream.
  * d/control:
    - Add python3-pyparsing as a new Build-Depends as this is required
      for the cherry picked fix.
    - Update the python-colorama and python-setuptools Built-Using versions.
    - Add pyparsing to Built-Using.
    - update-maintainer
  * d/rules: dirtbike up pyparsing.

 -- Barry Warsaw <email address hidden> Thu, 05 May 2016 16:11:04 -0500

[Chris J Arges (arges)]

The verification of the Stable Release Update for python-pip has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package pyparsing - 2.0.3+dfsg1-1ubuntu0.1

---------------
pyparsing (2.0.3+dfsg1-1ubuntu0.1) xenial; urgency=medium

  * d/patches/use-setuptools.patch: Use setuptools.setup() instead of
    distutils.setup() so an egg-info directory is generated instead of an
    egg-info file. (LP: #1578761)
  * d/control: update-maintainer

 -- Barry Warsaw <email address hidden> Fri, 06 May 2016 11:13:26 -0500