It would be nice if checkbox would tell me exactly what requirement was not met for a test that was not run.

Bug #855852 reported by Jeff Lane
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox
Fix Released
Low
Brendan Donegan

Bug Description

A test can have multiple requirements, and a given test run could have multiple tests that aren't run due to requirements not being satisfied.

It would be nice if, instead of a generic and unhelpful "Job requirements not met" checkbox would actually list WHICH requirement was not met.

A good example of this is the suspend/record_playback_after_suspend test. It has three requirements:

requires:
 device.category == 'AUDIO'
 package.name == 'sox'
 package.name == 'alsa-base'

and fails to run in Oneiric. Now I know the reason is that sox isn't installed. However, the user wouldn't know that. So having the test report actually say something like:

Requirement 'package.name == sox' not met

And please ignore the test itself, I only include that as an easy example.

Related branches

Jeff Lane (bladernr)
Changed in checkbox:
importance: Undecided → Low
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

Agree on the Low importance as it's only really useful for debugging. I can't see how this should be difficult since Checkbox is obviously going to be aware of which requirement wasn't met as it checks it.

Changed in checkbox:
status: New → Confirmed
status: Confirmed → Triaged
assignee: nobody → Brendan Donegan (brendan-donegan)
assignee: Brendan Donegan (brendan-donegan) → nobody
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

I found the place in the resource_info.py plugin where I think this should be done. Just trying to test it now.

Changed in checkbox:
assignee: nobody → Brendan Donegan (brendan-donegan)
status: Triaged → In Progress
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

The branch linked to this bug gives you a report like the one in the attachment, notice the difference ;)

Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

I'm still going to tidy up the change a bit to do nicer formatting where more than one requirement is not met (I have a feeling it will be a bit ugly with what's there at the moment)

Daniel Manrique (roadmr)
Changed in checkbox:
status: In Progress → Fix Committed
Revision history for this message
Ara Pulido (ara) wrote :

I think this won't be accepted, as it is a new feature.
Please, revert the changes and submit back to the checkbox_ui_rewrite

Thanks!

Changed in checkbox:
milestone: none → 0.13
status: Fix Committed → Triaged
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

Since checkbox/0.12 is branched now I'll resubmit this merge proposal to trunk

Changed in checkbox:
status: Triaged → In Progress
Changed in checkbox:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments