Super key, Mute Key, battery-info and volume keys test cases are failed always after suspend.

[Gabriel Zhi Chen]

Test machine CID: 201710-25818

Steps:
1. Install the latest checkbox on 201710-25818
2. Execute $ sutton-cli
3. Checked 'Hotkey tests' section
4. Press 'Enter' to start testing
5. Observe those test cases with '(S3)' tag in tail.

Expected result:
All test cases do the right behavior.

Actual result:
keys/media-control after suspend (S3);
keys/mute after suspend (S3);
keys/volume after suspend (S3);

Those test cases will got failed test results always.
In generally, when the cases executing, there should be a window popping up, but the window never appear.

*********************************
-------------[ Running job 46 / 49. Estimated time left: unknown ]--------------
-----------------------[ keys/super after suspend (S3) ]------------------------
ID: com.canonical.certification::after-suspend-manual-keys/super
Category: com.canonical.plainbox::keys
PURPOSE:
    This test will test the super key of your keyboard
STEPS:
    1. Click test to open a window on which to test the super key.
    2. If the key works, the test will pass and the window will close.
VERIFICATION:
    Does the super key work as expected?
Pick an action
    => press ENTER to continue
  c => add a comment
  s => skip this job
  q => save the session and quit
[csq]:
... 8< -------------------------------------------------------------------------
Cannot find job with checksum 0182d07df59e5ebf25fd5b5eb83ca1b037e04bb0929007ee0229f9b12642bb03
------------------------------------------------------------------------- >8 ---
Outcome: job failed
-------------[ Running job 47 / 49. Estimated time left: unknown ]--------------
------------------------[ keys/mute after suspend (S3) ]------------------------
ID: com.canonical.certification::after-suspend-manual-keys/mute
Category: com.canonical.plainbox::keys
PURPOSE:
    This test will test the mute key of your keyboard
STEPS:
    1. Click test to open a window on which to test the mute key.
    2. If the key works, the test will pass and the window will close.
VERIFICATION:
    Does the mute key work as expected?
Pick an action
    => press ENTER to continue
  c => add a comment
  s => skip this job
  q => save the session and quit
[csq]:
... 8< -------------------------------------------------------------------------
Cannot find job with checksum 36cc264c872a99999263fe647952c4f4aee61ce1bf61a77e8be2c0de049f2ab1
------------------------------------------------------------------------- >8 ---
Outcome: job needs verification
Please decide what to do next:
  outcome: job needs verification
  comments: none
Pick an action
  c => add a comment
  p => set outcome to pass
  f => set outcome to fail
  s => set outcome to skip
  r => re-run this job
    => set suggested outcome [job failed]
[cpfsr]:
-------------[ Running job 48 / 49. Estimated time left: unknown ]--------------
-----------------------[ keys/volume after suspend (S3) ]-----------------------
ID: com.canonical.certification::after-suspend-manual-keys/volume
Category: com.canonical.plainbox::keys
PURPOSE:
    This test will test the volume keys of your keyboard
STEPS:
    Skip this test if your computer has no volume keys.
    1. Click test to open a window on which to test the volume keys.
    2. If all the keys work, the test will be marked as passed.
VERIFICATION:
    Do the keys work as expected?
Pick an action
    => press ENTER to continue
  c => add a comment
  s => skip this job
  q => save the session and quit
[csq]:
... 8< -------------------------------------------------------------------------
Cannot find job with checksum 0535435d02d60020cf7645542a9e2feeaa805499baab4fa26117f4d6b450a5a4
------------------------------------------------------------------------- >8 ---
Outcome: job needs verification
Please decide what to do next:
  outcome: job needs verification
  comments: none
Pick an action
  c => add a comment
  p => set outcome to pass
  f => set outcome to fail
  s => set outcome to skip
  r => re-run this job
    => set suggested outcome [job failed]
[cpfsr]:
*****************************************

checkbox-converged 1.2.4-0ubuntu1
checkbox-gui 1.2.4-0ubuntu1
checkbox-ng 0.35.0~ppa~ubuntu16.04.1
checkbox-ng-service 0.23~ppa~ubuntu16.04.1
checkbox-oem-bug 1.12-1-ubuntu1
plainbox-provider-checkbox 0.40.0~ppa~ubuntu16.04.1
python3-checkbox-ng 0.35.0~ppa~ubuntu16.04.1
python3-checkbox-support 0.36.0~ppa~ubuntu16.04.1
plainbox 0.37.0~ppa~ubuntu16.04.1
plainbox-insecure-policy 0.37.0~ppa~ubuntu16.04.1
plainbox-provider-certification-client 0.32.0~ppa~ubuntu16.04.1
plainbox-provider-checkbox 0.40.0~ppa~ubuntu16.04.1
plainbox-provider-oem 0.46
plainbox-provider-oem-sutton 0.46
plainbox-provider-resource-generic 0.34.0~ppa~ubuntu16.04.1
plainbox-provider-tpm2 1.6.0~ppa~ubuntu16.04.1
python3-plainbox 0.37.0~ppa~ubuntu16.04.1

[Jerry Kao (jerry.kao)]

Can reproduce this issue on 201611-25202.
key/battery-info after suspend failed with the same error "Cannot find job with checksum"

[Sylvain Pineau (sylvain-pineau)]

This bug can only be reproduced and classic where both sudo and trusted launcher controllers can be used.

When providers are installed system wide (normal install from a ppa), pkexec is preferred over sudo.
The plainbox trusted launcher is the used to run jobs as root.

But it needs to know/discover all jobs as only job checksums are used for security reasons.
Problem is that siblings jobs were not added to this list.

The linked MR proposes a fix for just the trusted launcher.

[Jerry Kao (jerry.kao)]

As this issue affects all OEM projects test, raise importance to critical.

@Sylvain, could we have the fix landing to stable sooner?

[Sylvain Pineau (sylvain-pineau)]

@Jerry, a new rc (plainbox - 0.38.0~rc1~ppa~ubuntu16.04.1) is available in the testing ppa with a fix for this bug (I've only cherry-picks the fix commit).

Please help to validate if keys tests after suspend are working now.

I'll push it to stable when I get a +1

[Jerry Kao (jerry.kao)]

I have validated fixed on plainbox - 0.38.0~rc1~ppa~ubuntu16.04.1. All key tests before and after suspend can be run without issue.

+1 to promote it to stable.