Comment 3 for bug 1269782

On 16 January 2014 11:31, Mark Brown <email address hidden> wrote:

> | Unfortunately, there is currently no way to reliably identify the first
> | case because the name of the test does not have to relate to the
> | filename of the YAML file.
>
> | The filter cannot tell the difference - there are simply no results
> | which match the filter.
>
> As a user of this stuff I have no idea what a "filter" is...
>

FWIW:
https://validation.linaro.org/static/docs/filters-reports.html?highlight=filter

>
> | It may be possible to collate the lava_test_shell data in the result
> | bundle in such a way as to create a list of test definitions for that
> | job and then annotate each test_id if the job failed.
>
> | This could help the filter distinguish between tests which were not
> | requested and tests which were requested but failed to run.
>
> ...I think the above matches what I'd thought was going on here. I had
> thought that we were scheduling a bunch of existing testsuites that LAVA
> knows about to run.
>
> --
> You received this bug notification because you are a member of Linaro
> Validation Team, which is subscribed to LAVA Server.
> https://bugs.launchpad.net/bugs/1269782
>
> Title:
> Unclear indication in dashboard when test setup fails
>
> Status in LAVA Server:
> New
>
> Bug description:
> Looking at the image report for TC2 LSK at
> https://validation.linaro.org/dashboard/image-reports/linux-linaro-
> lsk-vexpress-tc2 I see that several tests such as gator are showing as
> '-' indicating that they have not been run but there is no indication
> as to why. Looking at the output from the job this is due to failures
> starting the testsuites for example:
>
> + lava-install-packages bootchart lsb-release pybootchartgui
> 0% [Working] Err http://ports.ubuntu.com saucy InRelease
>
> Err http://ppa.launchpad.net saucy InRelease
>
> Err http://ppa.launchpad.net saucy InRelease
>
> 0% [Working] Err http://ports.ubuntu.com saucy Release.gpg
> Could not resolve 'ports.ubuntu.com'
> Err http://ppa.launchpad.net saucy Release.gpg
> Could not resolve 'ppa.launchpad.net'
> 0% [Working] Err http://ppa.launchpad.net saucy Release.gpg
> Could not resolve 'ppa.launchpad.net'
>
> I would expect this to show up as red on the dashboard or to have a
> separate line item there for test prerequisites showing that some of
> the testsuites failed to set up, or perhaps the test setup should be
> included as a test within the test list for reporting. As things stand
> it was not at all clear to me looking at the dashboard that any
> attempt had been made to run the tests.
>
> The fact that the failures happened is a separate issue to the fact
> that this isn't reported clearly.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/lava-server/+bug/1269782/+subscriptions
>

--

Alan Bennett, Engineering Manager, Linaro LAVA Team
Linaro.org <http://www.linaro.org/> *│ *Open source software for ARM SoCs |
Follow Linaro*:* Facebook <http://www.facebook.com/pages/Linaro> |
Twitter<http://twitter.com/#%21/linaroorg>
 | Blog <http://www.linaro.org/linaro-blog/>
irc: akbennett | <email address hidden> |
<email address hidden>