Namespace packages in site-packages can't be found

[Barry Warsaw]

To reproduce:

sudo apt-get install zope.interface
bzr lp:mailman
cd mailman
(edit buildout.cfg to remove the include-site-packages line)
python bootstrap.py
bin/buildout

Here's the problem. I have python-zope.interface package installed on Ubuntu 10.10. Mailman requires zope.interface and several other zope.* packages, like .component, .configuration, and .testing. When I buildout the tree I get errors that 'zope' has no module 'interface', and I think the reason is that namespace packages aren't set up correctly between the builtout environment and the system's packages.

Specifically, I can see that those other zope packages get downloaded (the ones that aren't installed in site), and when I import zope from bin/py I can see the namespace package, but it apparently cannot find zope.interface, which is in dist-packages. Suppressing include-site-packages "fixes" the problem in that zope.interface in dist-packages is ignored and it's downloaded into my buildout environment.

[Gary Poster (gary)]

As I mentioned on IRC, this basic behavior is something for which buildout has several explicit tests, so it seems mailman has encountered another namespace package + system Python + buildout edge case that we don't have--maybe Maverick is doing something new. I'll try to dupe and fix.

[Alan Franzoni (alanfranz)]

Probably related:

https://bugs.launchpad.net/ubuntu/+source/zope.interface/+bug/688335

could you try removing the __init__.py* from your /usr/lib/python2.6/dist-packages/zope directory and verify the problem vanishes?

[Alan Franzoni (alanfranz)]

One more thing: could you check if this is what affects you instead?

https://bugs.launchpad.net/ubuntu/+source/distribute/+bug/576434

[Gediminas Paulauskas (menesis)]

Looks to me that this is caused by http://bugs.python.org/setuptools/issue113

zope packages for Debian are built with --single-version-externally-managed flag that creates a *-nspkg.pth file, and skips the zope/__init__.py file. But, we later install the __init__.py file anyway (zope.interface does this). But the nspkg.pth file does not work when the __init__.py exists.

Part of that file reads:

... ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('zope',types.ModuleType('zope')) ...

if I delete "not ie" clause from zope.interface-*-nspkg.pth, this problem goes away, because then '/usr/lib/python2.6/dist-packages/zope' is added to sys.modules['zope'].__path__ and all zope.* packages are found.

However, the install_namespaces function in setuptools.commands.install_egg_info that generates the nspkg.pth files has not been changed since it was first introduced in 0.6b3...