Tests Fail on saucy - lock_read missing from "Thing"

[David Britton]

======================================================================
ERROR: test_run (tarmac.plugins.tests.test_command.TestCommand)
tarmac.plugins.tests.test_command.TestCommand.test_run
----------------------------------------------------------------------
_StringException: log: {{{8.141 export version {'abspath': <function abspath at 0x7f0274c6e9b0>}}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/tests/test_command.py", line 49, in test_run
    proposal=self.proposal)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/command.py", line 108, in run
    per_file_timestamps=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 175, in export
    per_file_timestamps, fileobj):
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 139, in get_export_generator
    tree.lock_read()
AttributeError: 'Thing' object has no attribute 'lock_read'

======================================================================
ERROR: test_run_exported_tree (tarmac.plugins.tests.test_command.TestCommand)
tarmac.plugins.tests.test_command.TestCommand.test_run_exported_tree
----------------------------------------------------------------------
_StringException: log: {{{8.144 export version {'abspath': <function abspath at 0x7f0274c6e9b0>}}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/tests/test_command.py", line 70, in test_run_exported_tree
    proposal=self.proposal)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/command.py", line 108, in run
    per_file_timestamps=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 175, in export
    per_file_timestamps, fileobj):
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 139, in get_export_generator
    tree.lock_read()
AttributeError: 'Thing' object has no attribute 'lock_read'

======================================================================
ERROR: test_run_failure (tarmac.plugins.tests.test_command.TestCommand)
tarmac.plugins.tests.test_command.TestCommand.test_run_failure
----------------------------------------------------------------------
_StringException: log: {{{8.146 export version {'abspath': <function abspath at 0x7f0274c6e9b0>}}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/tests/test_command.py", line 60, in test_run_failure
    proposal=self.proposal)
  File "/usr/lib/python2.7/dist-packages/bzrlib/tests/__init__.py", line 1519, in assertRaises
    callableObj(*args, **kwargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/command.py", line 108, in run
    per_file_timestamps=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 175, in export
    per_file_timestamps, fileobj):
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 139, in get_export_generator
    tree.lock_read()
AttributeError: 'Thing' object has no attribute 'lock_read'

======================================================================
ERROR: test_run_nonascii_failure (tarmac.plugins.tests.test_command.TestCommand)
tarmac.plugins.tests.test_command.TestCommand.test_run_nonascii_failure
----------------------------------------------------------------------
_StringException: log: {{{8.149 export version {'abspath': <function abspath at 0x7f0274c6e9b0>}}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/tests/test_command.py", line 91, in test_run_nonascii_failure
    proposal=self.proposal)
  File "/usr/lib/python2.7/dist-packages/bzrlib/tests/__init__.py", line 1519, in assertRaises
    callableObj(*args, **kwargs)
  File "/home/dpb/src/canonical/tarmac/trunk/tarmac/plugins/command.py", line 108, in run
    per_file_timestamps=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 175, in export
    per_file_timestamps, fileobj):
  File "/usr/lib/python2.7/dist-packages/bzrlib/export/__init__.py", line 139, in get_export_generator
    tree.lock_read()
AttributeError: 'Thing' object has no attribute 'lock_read'

----------------------------------------------------------------------
Ran 89 tests in 8.007s

FAILED (errors=4)

Happens with latest code (revno=424) + bzr lib on saucy. I have not checked other releases, but I can build/test on precise to keep my branch moving along.

[dobey (dobey)]

Interestingly, these failures do not happen on 14.04. This seems to only happen on 13.10. I wonder what exactly is causing the problem. Since 13.10 isn't an LTS, and tarmac is not included in the Ubuntu archive currently, I'm lowering the priority on this to Medium.

13.10 will also be EOL this summer. I'd be happy to have a fix for it, but it seems like an issue caused by bzrlib perhaps. The tests fail, but tarmac itself should work still.

[David Britton (dpb)]

Hi --

I did a lot of investigation over the past 2 days and tracked this down to a module import isolation problem with the unittest runner. If you have an environment to reproduce the issue (for me trusty still hit it), running with:

  nosetests tarmac

or

  trial tarmac

would not hit the problem. Also, running just the failing tests in isolation does not hit the problem. For example:

  python -m unittest tarmac.plugins.tests.test_command.TestCommand #PASS
  python -m unittest discover #FAIL

This led me to find the other test (or tests) that were causing an issue. I eventually narrowed it down to the mocking library's "@patch" decorator, not working for the plugin that was under test (tarmac.plugins.command). Which led me to the plugin.load_plugins() test cases. One of those tests cases (test_load_plugins) actually did a delattr() on all loaded plugins both before the test, and after the test had executed.

I didn't root cause why `python -m unittest discover` hits this issue where trial and nose do not, but I chalked it up to better isolation native to those runners, which I've experienced in other various bugs on other projects. They also both publish that they have better isolation features.

The only other avenue I walked down was looking into python-mock. the `@patch` decorator claims to only make the patch exactly before the test runs, and revert it after, but perhaps the symbol table it's using is somehow corrupted by the `delattr()` call in the load_plugin() test. Anyway, checking into bugs in python-mock didn't prove fruitful.

The attached branch should solve the issue. It slightly changes test coverage in that it doesn't explicitly load each plugin, but that is taken care of in other places in the tests already (thus the reason it had to delattr(), they were already loaded). I think isolating it to explicitly load a dummy plugin is more in line with the intent of the unit test anyway. Given others were already doing that in the suite, I trust this should be uncontroversial to accept.

[David Britton (dpb)]

BTW, here is a rough testcase I used for validating on saucy and trusty

sudo lxc-create -t ubuntu -n trusty -- -r trusty -b $USER
sudo lxc-start -d -n trusty-nobind
sudo lxc-ls -f # get ip address
ssh $USER@$ip
sudo apt-get install python-setuptools python-mock python-bzrlib.tests
cd <tarmac-checkout-path>
./run-tests