One failing functional test prevents all following tests from running

[Jonathan Lange]

If one functional test fails, then none of the other functional tests will run. This is a problem because it means that test runs with failing functional tests will have only partial information, possibly hiding other failures which could be useful for debugging problems.

As Itamar says on https://code.launchpad.net/~zomux/twistedchecker/automaticallytest/+merge/108273:

  6. I suspect there's a better way to implement the automated test runner, so it integrates better with trial. E.g. right now it'll stop upon first failure; if each test case was a trial test you could see results of other tests. However, that doesn't seem worth blocking your progress. Jonathan may have suggestions if you later have time to come back and improve this. So sticking with this implementation seems OK to me for now.

One approach for solving this would be to mimic what pyflakes does. e.g. http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Pyflakes/pyflakes/test/test_other.py

This bug refers to code in https://code.launchpad.net/~zomux/twistedchecker/automaticallytest/+merge/108273 which is scheduled to land soon.