sleep_test_log_check should provide a more human-friendly output
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Checkbox Provider - Base |
Won't Fix
|
Wishlist
|
Sylvain Pineau |
Bug Description
When running S3/S4 stress tests, the test plans usually include a job to perform the actual suspend/resume (or hibernate/resume) and a job to check the log generated by these jobs (usually with id like `*-log-check`).
We recently launched S4 stress tests on a device [1], and got the following output:
High failures:
s4: 300 failures
=======
Found 1 differences in device configuation during S4 cycle. (x 99)
Found 2 differences in device configuation during S4 cycle. (x 1)
Failed to restore hibernate image. (x 100)
Failed to freeze devices. (x 100)
Given that we only had 100 iterations, it's very surprising for a reviewer to see "300 failures".
Since these output may be provided to customers, it would be better to have a more human-friendly output.
It was suggested to keep the current output, but to provide a higher level summary before, such as:
============ Test Summary ============
Total iterations: 100
Iterations passed: 0
Iterations failed: 100
============ Test Details ============
High failures:
s4: 300 failures
=======
Found 1 differences in device configuation during S4 cycle. (x 99)
Found 2 differences in device configuation during S4 cycle. (x 1)
Failed to restore hibernate image. (x 100)
Failed to freeze devices. (x 100)
[1] https:/
Changed in plainbox-provider-checkbox: | |
milestone: | none → 0.47.0 |
Changed in plainbox-provider-checkbox: | |
milestone: | 0.47.0 → 0.48.0 |
Changed in plainbox-provider-checkbox: | |
milestone: | 0.48.0 → none |
Changed in plainbox-provider-checkbox: | |
importance: | Medium → Wishlist |
Changed in plainbox-provider-checkbox: | |
status: | Confirmed → Won't Fix |