sleep_test_log_check should provide a more human-friendly output

Bug #1771762 reported by Pierre Equoy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox Provider - Base
Won't Fix
Wishlist
Sylvain Pineau

Bug Description

When running S3/S4 stress tests, the test plans usually include a job to perform the actual suspend/resume (or hibernate/resume) and a job to check the log generated by these jobs (usually with id like `*-log-check`).

We recently launched S4 stress tests on a device [1], and got the following output:

High failures:
  s4: 300 failures
========================================
    Found 1 differences in device configuation during S4 cycle. (x 99)
    Found 2 differences in device configuation during S4 cycle. (x 1)
    Failed to restore hibernate image. (x 100)
    Failed to freeze devices. (x 100)

Given that we only had 100 iterations, it's very surprising for a reviewer to see "300 failures".

Since these output may be provided to customers, it would be better to have a more human-friendly output.

It was suggested to keep the current output, but to provide a higher level summary before, such as:

============ Test Summary ============

Total iterations: 100
Iterations passed: 0
Iterations failed: 100

============ Test Details ============

High failures:
  s4: 300 failures
========================================
    Found 1 differences in device configuation during S4 cycle. (x 99)
    Found 2 differences in device configuation during S4 cycle. (x 1)
    Failed to restore hibernate image. (x 100)
    Failed to freeze devices. (x 100)

[1] https://certification.canonical.com/hardware/201712-26014/submission/130055/test-results/

Changed in plainbox-provider-checkbox:
milestone: none → 0.47.0
Changed in plainbox-provider-checkbox:
milestone: 0.47.0 → 0.48.0
Changed in plainbox-provider-checkbox:
milestone: 0.48.0 → none
Changed in plainbox-provider-checkbox:
importance: Medium → Wishlist
Changed in plainbox-provider-checkbox:
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.