Multiple test run information should be available in report

Bug #1419869 reported by Chris Gregan on 2015-02-09
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Driver Test Suite
Critical
Sylvain Pineau
PlainBox (Toolkit)
High
Zygmunt Krynicki

Bug Description

Submitted by nVidia:

Be able to rerun/retry each tests [0] and so it's possible to make all information into a single report.
Sometimes in PES have two or more submissions as references to explain why something fails at first place, but this is could be a trouble in between IHV and ODM due to different R&Rs involving. To have a retry record and logs in report sounds reasonable and a possible way to eliminate some unnecessary concerns for report reviewer.

An ideal use case will look like the following:

1. A complete CDTS run is done with some failing test items.
2. Testers get to resume the same test session and only focus on the failed test jobs.
3. Testers can resume the test session whatever number of times until they are content with the end results.
4. The resulting test report for the test session would have recorded all the test results for those re-attempted jobs.

Chris Gregan (cgregan) on 2015-03-26
Changed in cdts:
milestone: 1.16 → 1.15.1
milestone: 1.15.1 → 1.16
Zygmunt Krynicki (zyga) on 2015-04-23
Changed in plainbox:
assignee: nobody → Zygmunt Krynicki (zyga)
Chris Gregan (cgregan) on 2015-04-23
Changed in cdts:
milestone: 1.16 → 1.17
Zygmunt Krynicki (zyga) wrote :

This is a low hanging fruit. We can do this to just a few lines of python in the job state class.

Sylvain Pineau (sylvain-pineau) wrote :

The rerun first action is to set the job result to None IIRC, how will this preserve all results?

Store the real result in a new results tuple and make result the last
element of that tuple or a fake hollow memory result otherwise. Then
intercept assignment and just create a new tuple. Since this is a POD
it's pretty simple to do.

On Thu, Apr 23, 2015 at 3:19 PM, Sylvain Pineau
<email address hidden> wrote:
> The rerun first action is to set the job result to None IIRC, how will
> this preserve all results?
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1419869
>
> Title:
> Multiple test run information should be available in report
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cdts/+bug/1419869/+subscriptions

Chih-Hsyuan Ho (chih) on 2015-05-07
description: updated
Zygmunt Krynicki (zyga) on 2015-05-13
Changed in plainbox:
status: New → In Progress
importance: Undecided → High
milestone: none → 0.22
Zygmunt Krynicki (zyga) on 2015-06-16
Changed in plainbox:
status: In Progress → Fix Committed
Changed in cdts:
status: Confirmed → In Progress
Changed in cdts:
status: In Progress → Fix Committed
Changed in plainbox:
status: Fix Committed → Fix Released
Changed in cdts:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers