plainbox-provider-certification-client has dead entries in some whitelists

[Zygmunt Krynicki]

To reproduce this issue, go to the plainbox-provider-certification-client provider and run ./manage.py validate -N

(venv)zyga@cayuga:~/checkbox/providers/plainbox-provider-certification-client$ ./manage.py validate -N | grep error
whitelists/client-cert-12-04-4.whitelist:185: error: test plan 'client-cert-12-04-4', field 'include', selector '2013.com.canonical.certification::usb/usb3_read_performance' doesn't match any known or generated job
whitelists/client-cert-12-04-4.whitelist:214: error: test plan 'client-cert-12-04-4', field 'include', selector '2013.com.canonical.certification::suspend/display_after_suspend' doesn't match any known or generated job
whitelists/client-cert.whitelist:31: error: test plan 'client-cert', field 'include', selector '2013.com.canonical.certification::touchpad_scroll' doesn't match any known or generated job
whitelists/client-cert.whitelist:326: error: test plan 'client-cert', field 'include', selector '2013.com.canonical.certification::firmware/fwts_wakealarm' doesn't match any known or generated job
whitelists/client-cert.whitelist:328: error: test plan 'client-cert', field 'include', selector '2013.com.canonical.certification::firmware/fwts_uefirtvariable' doesn't match any known or generated job
whitelists/client-selftest-12-04-4.whitelist:185: error: test plan 'client-selftest-12-04-4', field 'include', selector '2013.com.canonical.certification::usb/usb3_read_performance' doesn't match any known or generated job
whitelists/client-selftest-12-04-4.whitelist:219: error: test plan 'client-selftest-12-04-4', field 'include', selector '2013.com.canonical.certification::suspend/display_after_suspend' doesn't match any known or generated job
whitelists/client-selftest.whitelist:28: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::touchpad_scroll' doesn't match any known or generated job
whitelists/client-selftest.whitelist:314: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_wakealarm' doesn't match any known or generated job
whitelists/client-selftest.whitelist:316: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_uefirtvariable' doesn't match any known or generated job

[Sylvain Pineau (sylvain-pineau)]

I'm working on fixing this bug but I don't really see why those jobs are reported as errors:

whitelists/client-selftest.whitelist:314: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_wakealarm' doesn't match any known or generated job
whitelists/client-selftest.whitelist:316: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_uefirtvariable' doesn't match any known or generated job

Both are generated by firmware/fwts and should be matched properly with the existing whitelists.
May be an issue with manage.py validate...

[Zygmunt Krynicki (zyga)]

W dniu 05.11.2014 o 14:05, Sylvain Pineau pisze:
> I'm working on fixing this bug but I don't really see why those jobs are
> reported as errors:
>
> whitelists/client-selftest.whitelist:314: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_wakealarm' doesn't match any known or generated job
> whitelists/client-selftest.whitelist:316: error: test plan 'client-selftest', field 'include', selector '2013.com.canonical.certification::firmware/fwts_uefirtvariable' doesn't match any known or generated job
>
> Both are generated by firmware/fwts and should be matched properly with the existing whitelists.
> May be an issue with manage.py validate...
>
Since fwts/wakealarm is not a pattern we don't have any hints to know it
might be generated hence the erorr is less helpful. I think that you are
right though I don't like the outcome. Before we can perfectly
understand local jobs and templates and have the derivative regular
expression engine we cannot detect errors like that reliably :-/

Thanks
ZK