"dist must be a Distribution instance" with pip 6 / setuptools >=10.0 and easy_install

[Ryan Campbell]

Recently (after upgrading to pip 6.0.6, on all my python projects that use pbr running 'python setup.py install/develop/etc..' results in the following error:

    (pbr-test)campbr9:~/git/pbr-test (master) $ python setup.py develop
    running develop
    running egg_info
    writing requirements to Test.egg-info/requires.txt
    writing Test.egg-info/PKG-INFO
    writing top-level names to Test.egg-info/top_level.txt
    writing dependency_links to Test.egg-info/dependency_links.txt
    writing pbr to Test.egg-info/pbr.json
    [pbr] Reusing existing SOURCES.txt
    running build_ext
    Creating /localhome/campbr9/.virtualenvs/pbr-test/lib/python2.7/site-packages/Test.egg-link (link to .)
    Test 0.1.0.dev1 is already the active version in easy-install.pth
    Installed /localhome/campbr9/git/pbr-test
    Processing dependencies for Test==0.1.0.dev1
    Searching for stevedore
    Reading http://example.com/prod/+simple/stevedore/
    Best match: stevedore 1.2.0
    Downloading http://example.com/+f/bd8/2728ab03e6212/stevedore-1.2.0.tar.gz#md5=bd82728ab03e62121fbbb077bcd55fdb
    Processing stevedore-1.2.0.tar.gz
    Writing /tmp/easy_install-_j8MPi/stevedore-1.2.0/setup.cfg
    Running stevedore-1.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-_j8MPi/stevedore-1.2.0/egg-dist-tmp-KsgYnU
    ERROR:root:Error parsing
    Traceback (most recent call last):
      File "/localhome/campbr9/git/pbr-test/.eggs/pbr-0.10.7-py2.7.egg/pbr/core.py", line 104, in pbr
        attrs = util.cfg_to_args(path)
      File "/localhome/campbr9/git/pbr-test/.eggs/pbr-0.10.7-py2.7.egg/pbr/util.py", line 240, in cfg_to_args
        kwargs = setup_cfg_to_setup_kwargs(config)
      File "/localhome/campbr9/git/pbr-test/.eggs/pbr-0.10.7-py2.7.egg/pbr/util.py", line 359, in setup_cfg_to_setup_kwargs
        cmd = cls(dist)
      File "/localhome/campbr9/.virtualenvs/pbr-test/local/lib/python2.7/site-packages/setuptools/__init__.py", line 124, in __init__
        _Command.__init__(self,dist)
      File "/usr/lib/python2.7/distutils/cmd.py", line 58, in __init__
        raise TypeError, "dist must be a Distribution instance"
    TypeError: dist must be a Distribution instance

I've created a minimal example project that can reproduce the problem, here (it's simply a empty project that uses pbr and depends on stevedore):

https://github.com/campbellr/pbr-bug

As mentioned in the README of that project, after doing some testing with various versions of setuptools, it seems like this started happening with setuptools 10.0.

Let me know if any more information is needed.

Ryan

[Doug Hellmann (doug-hellmann)]

I'm not able to recreate this with pip 6.0.8. Is it still causing you trouble?

[Ryan Campbell (rcampbell)]

Hi Doug,

Yeah, it looks like i still see this with pip 6.0.8.

In order to rule out any configuration-related issues I decided to try to reproduce the problem in a fresh docker container, and record the exact steps to reproduce.

I've attached the output as docker_pbr_output.txt

I hope that helps. I'd investigate further, but I can't really make heads or tails of all the setuptools/distutils/pbr magic that's going on...

[Robert Collins (lifeless)]

I think what is happening is that there are two copies of pkg_resources in memory and a Distribution object from the older copy is being passed to code doing a typecheck vs the newer copy. You'll need some gritty debugging to verify this.

The fix is to get declarative setup_requires supported by pip and stop the madness of easy_install kicking in at all.

You might be able to work around this by pip install -U pkg_resources before you do anything with pbr.

[Robert Collins (lifeless)]

(and/or pip install pbr)

[Anne Gentle (annegentle)]

I get this error with pip 7.10 and pbr 1.4.0.

[Gumpwa Luong (gumpwa)]

I also hit this problem. As a temporary work around I do the following:

'pip install -e .'

[Robert Collins (lifeless)]

So, this needs someone to break into pdb at the failure point and figure out whats going on :(.

If you hit it using tox, https://bitbucket.org/hpk42/tox/issues/320/fails-to-detect-changed-dependencies-and is probably the proximate cause- use tox -r to avoid that.

[Stephen Finucane (stephenfinucane)]

These versions of pip and setuptools are really old at this point and any fixes will likely benefit and increasingly small minority of users. As such, I'm going to close this WONTFIX. If someone really cares about this, feel free to re-open (ideally with suggestions on how to fix it).