Multiple test run information should be available in report

[Chris Gregan]

Submitted by nVidia:

Be able to rerun/retry each tests [0] and so it's possible to make all information into a single report.
Sometimes in PES have two or more submissions as references to explain why something fails at first place, but this is could be a trouble in between IHV and ODM due to different R&Rs involving. To have a retry record and logs in report sounds reasonable and a possible way to eliminate some unnecessary concerns for report reviewer.

An ideal use case will look like the following:

1. A complete CDTS run is done with some failing test items.
2. Testers get to resume the same test session and only focus on the failed test jobs.
3. Testers can resume the test session whatever number of times until they are content with the end results.
4. The resulting test report for the test session would have recorded all the test results for those re-attempted jobs.

[Zygmunt Krynicki (zyga)]

This is a low hanging fruit. We can do this to just a few lines of python in the job state class.

[Sylvain Pineau (sylvain-pineau)]

The rerun first action is to set the job result to None IIRC, how will this preserve all results?

[Zygmunt Krynicki (zyga)]

Store the real result in a new results tuple and make result the last
element of that tuple or a fake hollow memory result otherwise. Then
intercept assignment and just create a new tuple. Since this is a POD
it's pretty simple to do.

On Thu, Apr 23, 2015 at 3:19 PM, Sylvain Pineau
<email address hidden> wrote:
> The rerun first action is to set the job result to None IIRC, how will
> this preserve all results?
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1419869
>
> Title:
> Multiple test run information should be available in report
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cdts/+bug/1419869/+subscriptions