Redundant suspend test in I+N system test plan

[Jerry Kao]

[Summary]
When run somerville-iev-full test plan on I+N system (201803-26172), there are 2 suspend test after switch to discrete graphics (nvidia). Please refer following excerpt from test plan. Full test plan please refer to https://paste.ubuntu.com/p/b9N5wkV957/

com.canonical.certification::suspend/2_resolution_before_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::sleep
com.canonical.certification::suspend/2_suspend_after_switch_to_card_PCI_ID_0x1cbb_auto <--
com.canonical.certification::suspend/2_suspend-time-check_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_suspend-single-log-attach_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_compiz_check_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_driver_version_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_resolution_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_display_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_glxgears_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_video_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/2_cycle_resolutions_after_suspend_PCI_ID_0x1cbb_auto
com.canonical.certification::suspend/suspend_advanced <--
com.canonical.certification::after-suspend-manual-monitor/2_dim_brightness_PCI_ID_0x1cbb

com.canonical.certification::suspend/2_suspend_after_switch_to_card_PCI_ID_0x1cbb_auto [1] and com.canonical.certification::suspend/suspend_advanced are redundant. I am guessing after-suspend-manual-monitor/2_dim_brightness_PCI_ID_0x1cbb depends on suspend_advanced so it is there.

[1] suspend/2_suspend_after_switch_to_card_PCI_ID││_0x1cbb_auto is not an auto test case is another issue LP:1832493
[Steps to reproduce]

[Expected result]

[Actual result]

[Failure rate]

[Additional information]
system-manufacturer: Dell Inc.
system-product-name: Precision 5530
GPU: 00:02.0 VGA compatible controller: Intel Corporation Device 3e9b
kernel-version: 4.18.0-21-generic
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz (12x)
bios-version: 1.6.0

[Jerry Kao (jerry.kao)]

Automatically attached

[Jerry Kao (jerry.kao)]

Automatically attached

[Jerry Kao (jerry.kao)]

Automatically attached

[Jerry Kao (jerry.kao)]

Automatically attached

[Betty Lin (bettyl)]

I did some test on the daily build checkbox 0.49.
A lot of tests depend on suspend/advanced (for example: all the jobs which has flags - after-suspend-*), we should consider a method to cover it, then we won't get two suspend tests (suspend/x_suspend_after_switch_to_card_, suspend/advanced)

[Betty Lin (bettyl)]

Modify the manual suspend test
"suspend/{index}_suspend_after_switch_to_card_{product_slug}_auto" to
auto suspend. The reason we changed here is because in full test, if the
platform has both intergrated card and discreted card, it will run
manual suspend twice. And in current test plan
"after-suspend-graphics-integrated-gpu-cert-full", the first job
"graphics/1_auto_switch_card_.*" will reboot the machine and then it
runs two jobs "suspend/1_suspend-time-check_.*_auto" and
"suspend/1_suspend-single-log-attach_.*_auto" which are not running
after suspend. This modification can avoid the problem and user only run
manual suspend once.

[Betty Lin (bettyl)]

Verified manual suspend only occurred once with following checkbox version:

u@u-Inspiron-5391:~$ checkbox-cli --version
checkbox-ng: 1.5.0rc1
checkbox-support: 0.42.0rc1
com.canonical.ce:oem: 1.0
certification-client: 0.38.0rc1
plainbox-provider-checkbox: 0.49.0rc2
plainbox-provider-resource-generic: 0.41.0rc1
plainbox-provider-sru: 1.14.0rc1
plainbox-provider-tpm2: 1.11.0rc1