cannot use top level test discovery

[Robert Collins]

Mike pointed out to me that 'testtools.run discover -t . .' fails like so:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/site-packages/subunit/run.py", line 145, in <module>
    main()
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/site-packages/subunit/run.py", line 141, in main
    stdout=stdout, exit=False)
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 218, in __init
__
    self.parseArgs(argv)
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 257, in parseA
rgs
    self._do_discovery(argv[2:])
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 376, in _do_discovery
    loaded = loader.discover(start_dir, pattern, top_level_dir)
  File "/usr/lib/python2.7/unittest/loader.py", line 206, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 523, in _find_tests
    for test in pkg_tests:
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 523, in _find_tests
    for test in pkg_tests:
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 523, in _find_tests
    for test in pkg_tests:
  File "/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/testtools/run.py", line 492, in _find_tests
    raise ImportError(msg % (mod_name, module_dir, expected_dir))
ImportError: 'test_base' module incorrectly imported from '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy'. Expected '/home/robertc/work/openstack/oslo.db/oslo/db/sqlalchemy'. Is this module globally installed?

This is a bug - I'm not sure where, but opening it here as a starting point.

[Robert Collins (lifeless)]

http://paste.ubuntu.com/8400986/ is an import trace.

This is the import:
# /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_base.pyc matches /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_base.py
import oslo.db.sqlalchemy.test_base # precompiled from /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_base.pyc

note that it is being imported from the installed tree, not the test tree.

Walking up we can see that oslo.db itself is being imported from the installed tree:
import oslo.db # directory /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db
# /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db/__init__.pyc matches /home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db/__init__.py

(This means that if you edit code and just run tests, it won't run the new code!)

[Robert Collins (lifeless)]

Putting a breakpoint in at the installed oslo/db/__init__.py, sys.path:
['/home/robertc/work/openstack/oslo.db', '', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/plat-x86_64-linux-gnu', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/lib-tk', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/lib-old', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/site-packages']

So the source tree is first on the python path, which means we should find things there first (as expected).

[Robert Collins (lifeless)]

However oslo.db.__path__ is ['/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo/db'] (as we expect given the symptoms).

the path for olso is:
['/home/robertc/work/openstack/oslo.db/.tox/py27/local/lib/python2.7/site-packages/oslo', '/home/robertc/work/openstack/oslo.db/.tox/py27/lib/python2.7/site-packages/oslo']

And therein is the issue : we need the path for oslo to include '/home/robertc/work/openstack/oslo.db/oslo' first.

[Robert Collins (lifeless)]

So, the olso pth file that tox creates appends the installed tree's path into the oslo module __path__ at interpreter startup (because thats where .pth files are scanned and used).

[Robert Collins (lifeless)]

Ok, and the issue with pip install -e . is that no pth file is created (and addly the prior pth of the installed oslo.db is still present).

So it seems we have the following options:
 - teach pip install -e . to make a pth file [will help everyone developing setuptools namespace packages ?] and tell our devs to use that mode for running tests
 - after interpreter initialization alter the oslo module path - would probably require some directory-local metadata as the local code just isn't hit at all
 - stop using namespace packages (PEP 382 was rejected after all)
 - ???

[Robert Collins (lifeless)]

See also https://github.com/pypa/pip/issues/3

Another possible solution: add a .pth file. e.g.

site-packages/pbr-fixup.pth containing one line:
import pkg_resources; pkg_resources.fixup_namespace_packages('')

which fixes things for me.

[Robert Collins (lifeless)]

We could also switch to using

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

rather than pkg_resources, for less evil.

[Robert Collins (lifeless)]

Also http://legacy.python.org/dev/peps/pep-0420/ is essential reading. We could perhaps depend on importlib2 and use implicit namespace packages; a .pth file could cause importlib2 to be active whenever we've installed oslo.*.

[Robert Collins (lifeless)]

importlib2 is a bit broken at the moment, I've taken a quick stab at the coarse issues; will look a bit more deeply tomorrow.

[Viktor Serhieiev (vsergeyev)]

Command `python -m testtools.run discover -t . .` passes ok on oslo.db 2.0.0