Test that depend on unsupported test cases executed

Bug #509598 reported by Javier Collado
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox
Fix Released
Medium
Marc Tardif

Bug Description

When a test case depends on another test case for which requirements haven't been met, the test is executed anyway.

The expected output would be that none of the test cases that depend on the test cases that has failed is executed and that in the report all of them are marked as unsupported. The first one because the requirements haven't been met and the others because the dependencies have not been met.

Related branches

Revision history for this message
Javier Collado (javier.collado) wrote :
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

I have confirmed this using the file Javier provided. Logs are attached from such a run, but it is quite difficult to find out what's happening from the checkbox.log alone.

Revision history for this message
Brendan Donegan (brendan-donegan) wrote :
Changed in checkbox:
status: New → Confirmed
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

I think the importance should be low, unless someone can point out a use-case where this limitation is directly hurting them.

Revision history for this message
Daniel Manrique (roadmr) wrote :

I'm setting this to importance: Medium, as it's unreliable and unexpected behavior. A job *should* fail if its dependencies failed too.

Revision history for this message
Daniel Manrique (roadmr) wrote :

er, perhaps not fail, but "not run, as mentioned in the original report by Javier.

Changed in checkbox:
importance: Undecided → Medium
Revision history for this message
Marc Tardif (cr3) wrote :

It seems that tests are only skipped based on the requirements and those selected from the tree view. Otherwise, tests are never skipped based on dependencies. This is a regression which used to work but got refactored at some point. I'm linking a branch that attempts to fix the problem, enjoy!

Daniel Manrique (roadmr)
Changed in checkbox:
status: Confirmed → Fix Committed
assignee: nobody → Marc Tardif (cr3)
milestone: none → 0.13.1
Marc Tardif (cr3)
Changed in checkbox:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers