Comment 0 for bug 873241

This may just be a further issue with the decision to do the test multiplication during run(). (Wontfix?)

However, if I try to run my test suite using:
$ py -c "import sys; from testtools import run; import unittest; prog = run.TestProgram(testRunner=unittest.TextTestRunner(verbosity=2), stdout=sys.stdout)" discover

(so force testtools.run to actually be verbose about the tests being run), I see odd behavior.

If I just inherit from TestCaseWithScenario I see:
test__sync_exchange_updates_indexes (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test__sync_exchange_updates_indexes (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test_create_index (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test_create_index (u1db.tests.test_backends.DatabaseIndexTests) ... ok
...

If I change the code so that I inherit from testtools.TestCase, and I implement load_tests() as:
def load_tests(loader, standard_tests, pattern):
    suite = loader.suiteClass()
    suite.addTests(generate_scenarios(standard_tests))
    return suite

I then see:
u1db.tests.test_backends.DatabaseIndexTests.test__sync_exchange_updates_indexes (mem) ... ok
u1db.tests.test_backends.DatabaseIndexTests.test__sync_exchange_updates_indexes (sqlite) ... ok
u1db.tests.test_backends.DatabaseIndexTests.test_create_index (mem) ... ok
u1db.tests.test_backends.DatabaseIndexTests.test_create_index (sqlite) ... ok

Even weirder, if I inherit directly from unittest.TestCase instead of testtools.TestCase, I see:
test__sync_exchange_updates_indexes (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test__sync_exchange_updates_indexes (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test_create_index (u1db.tests.test_backends.DatabaseIndexTests) ... ok
test_create_index (u1db.tests.test_backends.DatabaseIndexTests) ... ok

It looks like this is because unittest.TextTestRunner has getDescription defined as:
    def getDescription(self, test):
        if self.descriptions:
            return test.shortDescription() or str(test)
        else:
            return str(test)
And the unittest.TestCase.shortDescription tries to return the docstring or None, while testtools.TestCase.shortDescription returns self.id().

So it looks like testscenarios.TestWithScenarios should either inherit from testtools.TestCase, or just provide def shortDescription(): return self.id().

I think inheriting from testtools.TestCase is reasonable, given that testscenarios already has a dependency on testtools.clone_test_with_new_id().