The .testr.conf file in heat uses a loop to discover tests under contrib. The multi-step discovery command means that if any one instance fails, that failure is ignored and the tests continue. The result is that any import error introduced may cause a test file, or whole set of test files, to be ignored instead of breaking the test suite. This was discovered while debugging a problem with the version of oslo.messaging that moves the code out of the namespace package, and can be reproduced by inserting gibberish into any of the test files in heat to cause the import to fail.
The loop in .testr.conf can be eliminated by adding __init__.py files in the structure of contrib (for example, contrib/__init__.py and contrib/rackspace/__init__.py cause the rackspace contrib directory to be scanned). You could also possibly write a more sophisticated wrapper script for the discovery command that watches for any errors.
