Test run is not continued between test suites even with -k in case of failures

[Ricardo Salveti]

From README:
If you want the test suite to continue when one error has been found, do:
 make -k check

While this work for the same test suite, it doesn't works between different test suites.

One example with the current image for Panda:

root@linaro-ubuntu-desktop:~/pwrmgmt_adding_default_option# lava-test run pwrmgmt
LAVA: Running shell command: 'cd pm-qa && make -k check '
...
LAVA: (stdout) ###
LAVA: (stdout) ### z_sanity_check:
LAVA: (stdout) ### ensure all the cpus are online when the tests are finished
LAVA: (stdout) ### ###
LAVA: (stdout) z_sanity_check.0: checking all cpu are back online... pass
LAVA: (stdout) make[1]: Leaving directory `/root/.local/share/lava_test/installed-tests/pwrmgmt/pm-qa/cpuhotplug'
LAVA: (stdout) make[1]: Entering directory `/root/.local/share/lava_test/installed-tests/pwrmgmt/pm-qa/cpuidle'
LAVA: (stdout) cc -g -Wall -pthread cpuidle_killer.c -o cpuidle_killer
LAVA: (stdout) ###
LAVA: (stdout) ### cpuidle_01:
LAVA: (stdout) ### test the cpuidle files are present in the sysfs
LAVA: (stdout) ### https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/QA/Scripts#cpuidle_01
LAVA: (stdout) ###
LAVA: (stdout) cpuidle_01.0: checking 'current_driver' exists... fail
LAVA: (stdout) ###
LAVA: (stdout) ### cpuidle_02:
LAVA: (stdout) ### Run a cpuidle program killer
LAVA: (stdout) ### https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/QA/Scripts#cpuidle_02
LAVA: (stdout) ###
LAVA: (stdout) cpuidle_02.0: checking cpuidle program runs successfully (120 secs)... Test duration exceeded
LAVA: (stdout) fail
LAVA: (stderr) make[1]: *** [cpuidle_02.log] Error 1
LAVA: (stdout) ###
LAVA: (stdout) ### cpuidle_03:
LAVA: (stdout) ### Run a cpuidle program killer and unplug one by one the cpus (except cpu0)
LAVA: (stdout) ### https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/QA/Scripts#cpuidle_03
LAVA: (stdout) ###
LAVA: (stdout) cpuidle_03.0/cpu0: skipping cpu0... skip
LAVA: (stdout) cpuidle_03.0/cpu1: checking cpuidle program runs successfully (120 secs)... pass
LAVA: (stderr) make[1]: Target `check' not remade because of errors.
LAVA: (stdout) make[1]: Leaving directory `/root/.local/share/lava_test/installed-tests/pwrmgmt/pm-qa/cpuidle'
LAVA: (stderr) make: *** [check] Error 2
LAVA: Command 'cd pm-qa && make -k check ' returned non-zero exit status 2
LAVA: run complete, result_id is 'pwrmgmt.2012-06-24T02:06:33Z'
ERROR: Some of test steps returned non-zero exit code

It stops while running the 'cpuidle' test suite, but it would be usefult to run the remaining test suites from thelist, like the 'sched_mc' one.

[Hongbo Zhang (hongbo-zhang)]

This is fixed by ignoring the current fail and then going on with the remaining tests.
Patch has been merged into the git server.