new update_excuses output too compact, hiding information

[Matthias Klose]

the new update_excuses hides the information for successful autopkg tests which
makes the page more compact. I dislike that when searching for failed autopkg
tests which failed for some triggers, but succeeded with others. With the old
output you were able to just grep update_excuses to look for those tests, which
currently doesn't work anymore. Please restore the old behavior, or if that's
considered to be too big, then show successful autopkg tests when there are also
ones which are failing with certain triggers.

[Olivier Tilloy (osomon)]

I personally find that new behaviour much better when reading through the page.

If the issue is with grepping the source of the page, then maybe the results of the successful tests could be added back, but not visible when rendered as HTML?

[Matthias Klose (doko)]

On 7/17/20 10:31 AM, Olivier Tilloy wrote:
> I personally find that new behaviour much better when reading through
> the page.
>
> If the issue is with grepping the source of the page, then maybe the
> results of the successful tests could be added back, but not visible
> when rendered as HTML?

to make it clear: I'm grepping update_excuses.html. what else should I grep?

[Iain Lane (laney)]

On Fri, Jul 17, 2020 at 08:31:00AM -0000, Olivier Tilloy wrote:
> I personally find that new behaviour much better when reading through
> the page.

Yeah, that's been my feeling too. It lets people see which issues remain
and not cognitively load them with information that isn't a problem.

> If the issue is with grepping the source of the page, then maybe the
> results of the successful tests could be added back, but not visible
> when rendered as HTML?

The yaml.xz contains all of the test results. This is a proposal, but
maybe it would be an idea to develop a client for that, if people want
different visibility to that which we're displaying on the html page.
Either a CLI script or something which generates a parallel webpage.

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

[Sebastien Bacher (seb128)]

The smaller version is a solid improvement in my opinion as well, now the page can be opened and is almost immediatly usable when before it was freezing firefox for 30 seconds and lagging a lot making almost impossible to select text to copy for example. I'm in favor of keeping the main report concise and add tooling to get the details out of the yaml for those who need those

[Matthias Klose (doko)]

On 7/17/20 11:24 AM, Sebastien Bacher wrote:
> The smaller version is a solid improvement in my opinion as well, now
> the page can be opened and is almost immediatly usable when before it
> was freezing firefox for 30 seconds and lagging a lot making almost
> impossible to select text to copy for example. I'm in favor of keeping
> the main report concise and add tooling to get the details out of the
> yaml for those who need those

being concise is different than hiding. It's not that you are reloading the
report every 10 seconds, so the loading delay should be bearable. It's only
updated every two hours, maybe if you are lucky, hourly.

[Marc Deslauriers (mdeslaur)]

We are using autopkgtests for our security PPA, and really liked seeing how many tests passed as an indicator of how well an update was tested.

While I do agree for the dev release only displaying the failed tests is an improvement, for our PPA, it is problematic.

Could we possibly generate a second "full" html page that can be used to get all test results?

[Steve Beattie (sbeattie)]

Yes, I second this from the Ubuntu Security team's perspective, to at least provide a second page with "full" results. I was just looking at openjdk-8 autopkgtests on xenial and was surprised at the single test result displayed, but couldn't remember the general state of autopkgtests in xenial for things openjdk-8 adjacent, so assumed the level of testing was weak back then.

Thanks.