Feature request: hooks to catch tests without proper cleanup
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Zope 3 |
Won't Fix
|
Medium
|
Unassigned | ||
zope.testing |
Won't Fix
|
Medium
|
Unassigned | ||
zope.testrunner |
Invalid
|
Undecided
|
Unassigned |
Bug Description
When a unit test does not clean up properly after itself (e.g. changes the component registry and fails to call placelessTearDown, or leaves uncommited objects lying around in a transaction) it can cause a completely unrelated test to fail later. When you have many tests, finding the broken one may be difficult.
It would help if the testrunner let us register a number of hooks to be run before each setUp and after each tearDown. Each hook could be an object with two methods (say, beforeSetUp and afterTearDown) taking the current test as an argument. A hook could take a snapshot of the global state (say, call getSite()) in beforeSetUp and compare it in afterTearDown. If there's a mismatch, you've identified a misbehaving test.
I have implemented this for the SchoolTool test runner a while ago. Unfortunatelly the code is GPLed and requests for a licencing change so far were unsuccessful. I am willing to reimplement the feature for the Zope 3 test runner if there's interest.
Changed in zope.testing: | |
importance: | Undecided → Medium |
Changed in zope3: | |
status: | New → Won't Fix |
+1 on the feature: sounds very useful. The situation with library-level schooltool code is quite a shame. Given the licensing issues, unfortunately, -1 on someone with potential license problems implementing it. Marius, maybe you could write a clear set of requirements on how to do this (i.e., without any code in it) that someone else could implement? I hear that approach around license problems has precedent (are there any IP lawyers in our midst to confirm or deny? ;-) )