Suspend and resume time shows N/A when running S3 stress

[Danny Hsu]

When running power-management/suspend_30_cycles_with_reboots, the suspend and resume time will shows "N/A" in the progress window.

ii checkbox-ng 1.3.0~ppa~ubuntu18.04.1 all CheckBoxNG test runner
ii checkbox-oem-bug 1.14-1-ubuntu1 all Checkbox OEM bug
ii plainbox-provider-checkbox 0.47.0~ppa~ubuntu18.04.1 amd64 CheckBox provider for PlainBox
ii python3-checkbox-ng 1.3.0~ppa~ubuntu18.04.1 all CheckBoxNG test runner (Python 3 library)
ii python3-checkbox-support 0.40.0~ppa~ubuntu18.04.1 all collection of Python modules used by PlainBox providers
ii plainbox-insecure-policy 1.3.0~ppa~ubuntu18.04.1 all policykit policy required to use CheckBoxNG (insecure version)
ii plainbox-provider-certification-client 0.36.0~ppa~ubuntu18.04.1 all Client Certification provider for Plainbox
ii plainbox-provider-checkbox 0.47.0~ppa~ubuntu18.04.1 amd64 CheckBox provider for PlainBox
ii plainbox-provider-oem 0.62 all plainbox oem provider
ii plainbox-provider-oem-somerville 0.62 all plainbox oem provider for somerville
ii plainbox-provider-resource-generic 0.39.0~ppa~ubuntu18.04.1 amd64 CheckBox generic resource jobs provider
ii plainbox-provider-sru 1.13.0~ppa~ubuntu18.04.1 all CheckBox SRU jobs provider
ii plainbox-provider-tpm2 1.9.0~ppa~ubuntu18.04.1 all TPM 2.0 provider for PlainBox

[Jerry Kao (jerry.kao)]

Failed test case is power-management/suspend-30-cycles-time-check-with-reboots

[Pierre Equoy (pieq)]

I tried to run the newly created “30 suspend/resume cycles and 1 poweroff, 3 times” tests [1] on a test device and ran into similar issue:

--------------[ Running job 3 / 3. Estimated time left: 0:00:01 ]---------------
[ 30 suspend/resume cycles and 1 poweroff, 3 times (check logs for timing issues) ]
ID: com.canonical.certification::power-management/suspend-30-cycles-time-check-with-coldboots
Category: com.canonical.plainbox::stress
... 8< -------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/nest-2odcydkh.4ad38fbc8c6cbad5fc5a0cbf97a6f45419f2ad609f25b34045d3fe8f4d75f781/sleep_time_check", line 75, in <module>
    sys.exit(main())
  File "/tmp/nest-2odcydkh.4ad38fbc8c6cbad5fc5a0cbf97a6f45419f2ad609f25b34045d3fe8f4d75f781/sleep_time_check", line 45, in main
    sleep_time = float(line.split(':')[1].strip())
ValueError: could not convert string to float: 'N/A'
------------------------------------------------------------------------- >8 ---
Outcome: job failed

It seems that the reported results are incorrect in the logs to begin with:

2019-02-27 18:12:39,366 INFO ==================== Test Results ====================
Average time to sleep: N/A
Average time to resume: N/A
High Failures: 1
 WARNING: The following test cases were reported as high
 level failures by fwts. Please review the log at
 /home/u/.cache/plainbox/sessions/checkbox-cli-2019-02-27T09.42.28.session/CHECKBOX_DATA/fwts.log for more information.
  - sleep

See attached session for more information.

[1] https://code.launchpad.net/~pieq/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/362858