Cannot run testr tests

[Jonathan Lange]

When I run the testr tests using testr, with the latest trunk of testresources, testscenarios and fixtures, I get many, many errors. Here's a representative one:

======================================================================
ERROR: testrepository.tests.repository.test_file.TestFileRepository.test_initialise
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jml/src/testresources/trunk/lib/testresources/__init__.py", line 688, in tearDownResources
    tearDownResources(self, self.resources, _get_result())
  File "/home/jml/src/testresources/trunk/lib/testresources/__init__.py", line 710, in tearDownResources
    resource[1].finishedWith(getattr(test, resource[0]), result)
AttributeError: 'TestFileRepository' object has no attribute 'tempdir'

I don't know why this is happening and a quick debug revealed nothing.

[Robert Collins (lifeless)]

Thanks for noticing this!

I hacked in full frame reporting (btw that seems broken, I'll file a separate bug on testtools about it - its not showing TestCase.run(), the runner frames or anything).

I see this:
ERROR: testrepository.tests.commands.test_list_tests.TestCommand.test_no_config_file_errors
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robertc/source/unittest/testtools/working/testtools/runtest.py", line 169, in _run_user
    return fn(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/testresources/__init__.py", line 648, in tearDownResources
    tearDownResources(self, self.resources, _get_result())
  File "/usr/lib/pymodules/python2.7/testresources/__init__.py", line 670, in tearDownResources
    resource[1].finishedWith(getattr(test, resource[0]), result)
AttributeError: 'TestCommand' object has no attribute 'tempdir'

which helps a little, but not enough, because I can't tell where the call came from.

pdb tells me this, which is considerably more helpful:
  /usr/lib/python2.7/runpy.py(162)_run_module_as_main()
-> "__main__", fname, loader, pkg_name)
  /usr/lib/python2.7/runpy.py(72)_run_code()
-> exec code in run_globals
  /home/robertc/source/unittest/testtools/working/testtools/run.py(332)<module>()
-> main(sys.argv, sys.stdout)
  /home/robertc/source/unittest/testtools/working/testtools/run.py(329)main()
-> program = TestProgram(argv=argv, testRunner=runner, stdout=stdout)
  /home/robertc/source/unittest/testtools/working/testtools/run.py(173)__init__()
-> self.runTests()
  /home/robertc/source/unittest/testtools/working/testtools/run.py(322)runTests()
-> self.result = testRunner.run(self.test)
  /home/robertc/source/unittest/testtools/working/testtools/run.py(46)run()
-> return test.run(result)
  /usr/lib/python2.7/unittest/suite.py(103)run()
-> test(result)
  /usr/lib/python2.7/unittest/suite.py(65)__call__()
-> return self.run(*args, **kwds)
  /usr/lib/pymodules/python2.7/testresources/__init__.py(245)run()
-> test(result)
  /usr/lib/python2.7/unittest/case.py(385)__call__()
-> return self.run(*args, **kwds)
  /home/robertc/source/unittest/testtools/working/testtools/testcase.py(523)run()
-> return self.__RunTest(self, self.exception_handlers).run(result)
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(74)run()
-> return self._run_one(actual_result)
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(88)_run_one()
-> return self._run_prepared_result(ExtendedToOriginalDecorator(result))
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(100)_run_prepared_result()
-> self._run_core()
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(135)_run_core()
-> self._run_cleanups, self.result):
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(169)_run_user()
-> return fn(*args, **kwargs)
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(154)_run_cleanups()
-> function, *arguments, **keywordArguments)
  /home/robertc/source/unittest/testtools/working/testtools/runtest.py(173)_run_user()
-> return self._got_user_exception(sys.exc_info(...

[Robert Collins (lifeless)]

notes:
the testresources.run is in OptimisingTestSuite

the test attributes at time of failure:
 self.__dict__.keys()
['exception_handlers', '_testMethodName', '_TestCase__details', '_resultForDoCleanups', '_unique_id_gen', '_traceback_id_gens', '_cleanups', '_type_equality_funcs', '_TestCase__exception_handlers', '_TestCase__setup_called', '_testMethodDoc', '_TestCase__teardown_called', '_TestCase__RunTest']

we're in run-core's cleanup dispatch loop

self.case._cleanups is empty - so this is the last case blowing up.

[Robert Collins (lifeless)]

before the cleanup
[(<unbound method ResourcedTestCase.tearDownResources>, (<testrepository.tests.test_commands.TestFindCommand.test_looksupcommand id=0x18b8190>,), {})]

the cleanup failure causes a failed state - the tests haven't failed until then.

[Robert Collins (lifeless)]

before run_test_method:
self.case.__dict__.keys()
['exception_handlers', '_testMethodName', '_TestCase__details', '_resultForDoCleanups', '_unique_id_gen', '_traceback_id_gens', '_cleanups', '_type_equality_funcs', 'cmd', '_TestCase__exception_handlers', '_TestCase__setup_called', '_testMethodDoc', '_TestCase__teardown_called', '_TestCase__RunTest']

after:
(Pdb) self.case.__dict__.keys()
['exception_handlers', '_testMethodName', '_TestCase__details', '_resultForDoCleanups', '_unique_id_gen', '_traceback_id_gens', '_cleanups', '_type_equality_funcs', 'cmd', '_TestCase__exception_handlers', '_TestCase__setup_called', '_testMethodDoc', '_TestCase__teardown_called', '_TestCase__RunTest']

after tearDown:

(Pdb) self.case.__dict__.keys()
['exception_handlers', '_testMethodName', '_TestCase__details', '_resultForDoCleanups', '_unique_id_gen', '_traceback_id_gens', '_cleanups', '_type_equality_funcs', '_TestCase__exception_handlers', '_TestCase__setup_called', '_testMethodDoc', '_TestCase__teardown_called', '_TestCase__RunTest']

-> cmd is gone. So tearDown *and* cleanUp are both removing the cmd attribute.

[Robert Collins (lifeless)]

Some crazy code here:

class ResourcedTestCase(TestCase, testresources.ResourcedTestCase):
    """Make all testrepository tests have resource support."""

    def setUp(self):
        TestCase.setUp(self)
        testresources.ResourcedTestCase.setUpResources(self)
        self.addCleanup(testresources.ResourcedTestCase.tearDownResources,
            self)

I don't know WTH this subclasses ResourcedTestCase... *AND* does addCleanup.

I speculate it -may- have worked in the past due to testtools not using super.

[Robert Collins (lifeless)]

Ah - because of this:
  File "testrepository/tests/__init__.py", line 28, in setUp
    testresources.ResourcedTestCase.setUpResources(self)
TypeError: unbound method setUpResources() must be called with ResourcedTestCase instance as first argument (got TestCLIUI instance instead)

[Robert Collins (lifeless)]

Dropping the custom setUp fixes it, so I'm doing that.