The parallel testing had a test run that only reported about 3000 tests even though the full 16000+ ran. This was because the stream it processed included the following test: http://pastebin.ubuntu.com/976408/
Obviously, """No handlers could be found for logger "lazr.smtptest"\n""" has been inserted into the stream. Without that, the stream can be processed correctly.
This bug's title partially directs a solution by attributing responsibility. This comes from a discussion with Robert in which he said that subunit shouldn't be expected to handle malformed output of this sort. Similarly, we could look for all sources of these complaints in the Launchpad tests (probably from a __del__ method when something is being garbage collected) but that will be error-prone, difficult, and not future-proof.
Instead, the test runner itself ought to isolate the subunit output. We could do that either by making the subunit stream not go to stdout, or make everything else not go to stdout (e.g., "sys.stdout = sys.stderr"). Robert notes that the former solution has the disadvantage that it does not allow piping of subunit output, which is intended to be one of the primary user interfaces for the stream. To pursue the latter solution, we may have test breakage, now and in the future, but it preserves the subunit interface and he prefers it. He points out that diagnosis of future test breakage might be helped if we can turn the sys.stdout redirection to stderr off and on.
I'd prefer that the --subunit flag to the testrunner imply that the stdout isolation should occur; that is, by default when you ask for a subunit stream, you should get a clean subunit stream, with the stdout->stderr redirection for everything else writing to stdout from the tests. For debugging, you can simply not run with --subunit, which will not perform the redirection of stdout. Further, if you really want to use --subunit but disable the stdout/stderr switch, maybe we could have another switch (--disable-stdout-switch?) but I think this is a YAGNI for now: not using --subunit seems sufficient at this stage. Robert's not sure he agrees, but is choosing not to weigh in.
This change should be made in our zope.testing fork.
Another idea from Robert, which would be cool: "make the replacement sys.stdout grab a traceback and report the object triggering the write--e.g. a wrapper around sys.stderr with a custom write() + writelines() methods"