One failing functional test prevents all following tests from running

Bug #1010392 reported by Jonathan Lange on 2012-06-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
twistedchecker
Fix Committed
Undecided
Unassigned

Bug Description

If one functional test fails, then none of the other functional tests will run. This is a problem because it means that test runs with failing functional tests will have only partial information, possibly hiding other failures which could be useful for debugging problems.

As Itamar says on https://code.launchpad.net/~zomux/twistedchecker/automaticallytest/+merge/108273:

  6. I suspect there's a better way to implement the automated test runner, so it integrates better with trial. E.g. right now it'll stop upon first failure; if each test case was a trial test you could see results of other tests. However, that doesn't seem worth blocking your progress. Jonathan may have suggestions if you later have time to come back and improve this. So sticking with this implementation seems OK to me for now.

One approach for solving this would be to mimic what pyflakes does. e.g. http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Pyflakes/pyflakes/test/test_other.py

This bug refers to code in https://code.launchpad.net/~zomux/twistedchecker/automaticallytest/+merge/108273 which is scheduled to land soon.

Related branches

Jonathan Lange (jml) on 2012-06-08
description: updated
HawkOwl (hawkowl) on 2013-09-04
Changed in twistedchecker:
status: New → Confirmed
HawkOwl (hawkowl) on 2013-10-23
Changed in twistedchecker:
status: Confirmed → Fix Committed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers