HTML formatter

Bug #426283 reported by Jelmer Vernooij on 2009-09-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
subunit
Wishlist
Unassigned

Bug Description

Right now the Samba buildfarm has a couple of regular expressions that happen to match the pattern of subunit that the Samba testsuite outputs. It would be nice if we could replace this with a proper subunit formatter at some point, perhaps one that can optionally only list the test failures.

Jelmer Vernooij (jelmer) wrote :

(no need for Samba for this to be in perl btw)

Changed in subunit:
status: New → Triaged
importance: Undecided → Wishlist
tags: added: html samba
Robert Collins (lifeless) wrote :

What would it look like? Whipping up something trivial in simpletal should be a 10 minute job, if you can sketch out what you want the output to look like. Or possibly better yet, write a separate project formatter like I did in python-junitxml (and I'd be happy to write that for you).

http://tungwaiyip.info/software/sample_test_report.html is an example output from http://tungwaiyip.info/software/HTMLTestRunner.html, which sadly doesn't obey the unittest protocol - it fiddles stdout/stderr rather than depending on the TestResult protocol.

I'm dropping a mail to TIP in case someone there has an existing result object that would suite.

Robert Collins (lifeless) wrote :

Another option methoned on the list TIP list is to use an xslt transform + subunit2junitxml.

e.g.
make check | subunit2junitxml -o results.xml
xslt --whatever < results.xml > foo.html

Robert Collins (lifeless) wrote :

Oh, and another conceptually cool way would be to just write a javascript parser for subunit; then render just-in-time.

Jelmer Vernooij (jelmer) wrote :

Actually, the output from http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the thing we're looking for.

What I would like to replicate is something that works at least as well as this:

http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc

but preferably something nicer and with more overview.

On Sun, 2009-09-27 at 14:22 +0000, Jelmer Vernooij wrote:
> Actually, the output from
> http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the
> thing we're looking for.
>
> What I would like to replicate is something that works at least as well
> as this:
>
> http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc
>
> but preferably something nicer and with more overview.

So, basically this looks like:
 - pick a template engine
 - pick a JS UI library
 - write your page

The glue code required is quite trivial - you can see how small
pyjunitxml is.

-Rob

Jelmer Vernooij (jelmer) wrote :

On Sun, 2009-09-27 at 22:37 +0000, Robert Collins wrote:
> On Sun, 2009-09-27 at 14:22 +0000, Jelmer Vernooij wrote:
> > Actually, the output from
> > http://tungwaiyip.info/software/HTMLTestRunner.html would be sortof the
> > thing we're looking for.
> >
> > What I would like to replicate is something that works at least as well
> > as this:
> >
> > http://build.samba.org/?function=View+Build;host=bnhtest;tree=samba_4_0_test;compiler=gcc
> >
> > but preferably something nicer and with more overview.
>
> So, basically this looks like:
> - pick a template engine
> - pick a JS UI library
> - write your page
>
> The glue code required is quite trivial - you can see how small
> pyjunitxml is.
I'll have a look at this next time I play with the Samba buildfarm.

One of the next steps would be to display HTML with statistics about
these various tests over different test runs. This might need a helper
Python module that's not specific to HTML though.

Some examples of things this could display would be: what tests are
flapping, what tests fail across all buildfarm hosts (as opposed to one
that might be broken), what tests break regularly (and might be good to
run first or as part of "make quicktest"), etc.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden> - http://samba.org/~jelmer/
Jabber: <email address hidden>

Robert Collins (lifeless) wrote :

So to do flapping needs historical view - thats more a test repository problem than simply 'format a single test result'.

I agree that that is good to have, and one advantage of subunit is the ability to use a stock python based test repository. I'm open to writing a dedicated one if thats what you need, but I suspect that it isn't a subunit problem, per se.

Robert Collins (lifeless) wrote :

So, I've written a stock python based test repository. Someone needs to write/suck in the HTML formatting.

Jelmer Vernooij (jelmer) wrote :

OpenStack has one of these, would it perhaps be an option to just ship that with subunit?

https://docs.openstack.org/developer/os-testr/subunit2html.html

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers