Return values from tests are ignored

[Gavin Panella]

We've had a situation in Launchpad where tests return Deferred objects, but the test runner didn't care and we had not set run_tests_with to AsynchronousDeferredRunTest. These tests were silently passing.

Tests do not normally return anything, so it may be useful if testtools could warn about tests that do return something other than None, to draw attention to mistakes of this kind.

[Jonathan Lange (jml)]

I'm not 100% sure that this would be a desirable feature for core testtools.

However, if you're interested in doing this for Launchpad, you could do this override in your base TestCase:

  def _run_test_method(self, result):
        ret = self._get_test_method()()
        self.assertIs(None, ret)
        return ret

The API isn't public, so it's liable to change without concern for backwards compatibility. However, it would do the job.

This is also the first actual use case I've seen for bug 584824.

[Jonathan Lange (jml)]

Oh, yeah, that would of course fail when you used ADRT, so you'd probably want to override RunTest._run_core or something and then set a default run_tests_with on your base test case.