TypeError: dist must be a Distribution instance

[clayg]

So we tried filing a bug against setuptools:

https://bitbucket.org/pypa/setuptools/issue/73/

But we are still wrought with grief:

https://bugs.launchpad.net/swift/+bug/1217288

I believe the issues manifests itself with "older" versions of setuptools - like you get if you installed python-setuptools on Lucid or Precise.

A patch was identified against setuptools that seemed to work around the issue:

http://paste.openstack.org/raw/45403/

But the setuptools maintainer thought it was too heavy handed, and suggested the problem lie here:

https://github.com/openstack-dev/pbr/blob/master/pbr/core.py#L52

The following Vagrantfile seems to reliably demonstrate the issue:

    VAGRANTFILE_API_VERSION = "2"

    Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
      config.vm.box = "precise"
      config.vm.box_url = "http://files.vagrantup.com/precise64.box"
      config.vbguest.auto_update = false

      config.vm.provision :shell, :inline => "sudo apt-get update"
      config.vm.provision :shell, :inline => "sudo apt-get install git build-essential python-dev libffi-dev python-setuptools -y"
      config.vm.provision :shell, :inline => "[[ -e swift ]] || git clone https://github.com/openstack/swift.git"
      config.vm.provision :shell, :inline => "cd swift && sudo python setup.py develop"
    end

Normally the work-arounds I've seen just require getting a new-enough-version of pip and setuptools - but I've seen even veteran pythonistias claim they still had struggles after trying various work-arounds.

Is there any hope this problem could be addressed with a patch to pbr so common workflows don't encounter this roadblock?

[Kevin McCarthy (me-kevinmccarthy)]

This bug has been driving me crazy for months. There is quite a bit of work going on with setuptools recently so this bug often gets written off as "update your setuptools". The problem is that pbr is unpatching distribute, which setuptools has patched:

https://github.com/openstack-dev/pbr/blob/master/pbr/core.py#L52

This makes it look like a setuptools issue, when it's actually a pbr issue. Any package that includes pbr anywhere in its dependency tree gets bitten by this bug, so it's sending ripples of problems even outside the openstack ecosystem, for example:

https://github.com/Netflix/aminator/issues/147
https://bitbucket.org/dhellmann/virtualenvwrapper/issue/199/packaging-problem-unable-to-use
https://github.com/wal-e/wal-e/issues/65
https://bugs.launchpad.net/swift/+bug/1217288
https://bugzilla.redhat.com/show_bug.cgi?id=1048458
http://sourceforge.net/p/imdbpy/mailman/imdbpy-help/
https://code.google.com/p/sqlalchemy-migrate/issues/detail?id=165
http://www.mediawiki.org/wiki/Talk:Continuous_integration/Jenkins_job_builder

[Kevin McCarthy (me-kevinmccarthy)]

OK, so I think I have figured out what's going on here. pbr is monkeypatching part of setuptools (the Distribution instance) to be the old Distribution from distribute instead of setuptools. This works OK for PBR, but it *never sets it back*. When the next packages gets installed, it will break because it's trying to use the monkeypatched broken distribute Distribution instance instead of the actual setuptools Distribution instance. This explains some odd behavior, for instance:

1. If you pip install pbr and then run your setup.py install, it works. That's because the monkeypatch is undone at that point.
2. If you change the order of installation in your setup.py so pbr runs last it works.
3. If you run the setup.py in an environment where pbr already exists, it works since it doesn't need to actually install pbr, so the Distribute instance never gets monkeypatched.

I think these are reasons this is so hard to reproduce!

Here is the patch: https://review.openstack.org/#/c/101785/

BTW, submitting a patch for this seems needlessly complex. It took me a whole Sunday afternoon to figure it out! Let me know if I did anything wrong, it obviously means a lot about me to get this done.

[Jeremy Stanley (fungi)]

Thanks for finding this! And yes, our patch submission workflow is tailored to high-volume projects with full-time developers, but as a result the ramp-up for a first-time contributor is pretty involved (to the point where it ends up discouraging a lot of drive-by fixes).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/101785
Committed: https://git.openstack.org/cgit/openstack-dev/pbr/commit/?id=b07a50bfddda1cbb10faa64dc096ca9d855c483c
Submitter: Jenkins
Branch: master

commit b07a50bfddda1cbb10faa64dc096ca9d855c483c
Author: Kevin McCarthy <email address hidden>
Date: Sun Jun 22 14:33:58 2014 -1000

    Restore Monkeypatched Distribution Instance

    pbr is monkeypatching the Distribution intstance in setuptools, which is
    OK I guess, but then it never puts it back how it left it. As a
    consequence, whenever pbr installs before another project in a setup.py,
    it breaks setuptools for everything that comes after it. This patch
    will restore the Distribution instance.

    Closes-Bug: 1324784

    Change-Id: I20fbf255c634685e82f7b11987d2725de8280b9d