Comment 6 for bug 1269782

| Yes, but each scheduling operation is open to anyone to re-use
| testsuites in different LAVA jobs. Therefore, more results can exist
| for the filter to match. (Filters are the database queries behind the
| image reports - the filter collates the test suite results into sets
| which provide the data for the reports.)

I think the biggest UX issue I'm having here is that I'm having a hard time matching this and therefore the conclusions you draw from it with what I'm looking at in the UI. What I'm doing is going to an image report like:

https://validation.linaro.org/dashboard/image-reports/linux-linaro-lsk-vexpress-tc2

There I can see a list of builds which if I click through are linked to specific jobs that LAVA ran. This means that LAVA knows exactly what testsuites were run in that job (since it was what ran them) which in turn means that it should be able to tell me if some of them generated errors and bombed out during their setup phase.

More generally LAVA is the one running jobs so it really ought to know what testsuites it tried to run.

| Your specific bunch of existing testsuites may not change (often or at
| all) but most sets change frequently. There will always be situations
| where some tests expected to be in any one filter will simply be
| omitted from the submission by the user.

Sure, but that doesn't mean that if a testsuite is run and then fails during the environment setup then that information should be discarded.

If I were doing this by searching for results of a given testsuite what you're saying would be a bit easier to relate to but the UI I'm going through shows results organised by job.