Comment 6 for bug 996729

Notes from investigation:

The four runs have zope:layer tags inappropriately applied to about 50 tests, not layers.

This accounts for all of the discrepancies for the bigger three test runs.

The smaller test run also turns out to be another incident of bug 994752 (which is concerning, given that we have a ten minute time out for starting an ephemeral container at this point, but pretty clearly unrelated to the zope.testing p9 changes). worker-15 is absent from the results.

So, the incorrect zope:layer tags are the immediate source of the problem. We could switch to filtering on the test name not ending with ":setUp" and ":tearDown" as a workaround, but that's not a good resolution. We're trying to figure out how the zope.testing changes could have triggered this kind of change. It's far from obvious at the moment.

I am curious as to what subunit issues William saw, and I've asked him on IRC to clarify when he is back online. If he saw something else then perhaps we haven't fully identified the problem yet.