warm-reboot and cold-reboot jobs are too permissive

[Jonathan Cave]

There are a two sets of boot stress tests (warm & cold) that generate a number of checkbox job iterations to test that key devices remain discoverable to the system across a large number of device reboots.

The jobs depend on tools lspci, nmcli, lsub but does not have any requires fields requiring their presence.

In the case that the tools are not present the test also does not fail which means the intent of the test is not satisfied. These tests should be updated to correct these faults.

Test locations:

https://git.launchpad.net/plainbox-provider-checkbox/tree/units/stress/boot.pxu#n122
https://git.launchpad.net/plainbox-provider-checkbox/tree/units/stress/boot.pxu#n170

Example run show passed tests despite NetworkManager not being present:

https://certification.canonical.com/hardware/202203-30047/submission/273414/test-results/

[Maksim Beliaev (beliaev-maksim)]

Bug was migrated to GitHub: https://github.com/canonical/checkbox/issues/246.
Bug is no more monitored here.