very very slow parsing full bzr test runs

[Robert Collins]

10 minutes so far, hasn't completed - running it with e.g. .testrepository/11 where that is a full test run of bzr.

Its not the subunit parser itself, as that can parse the full thing in about 20 seconds.

[Martin Pool (mbp)]

Yes I think there's some problem in how it uses gio.

[Robert Collins (lifeless)]

Just noticed, it says '/home/robertc/source/tribunal/tribunal/eventloop.py:19: Warning: poll(2) failed due to: Invalid argument.
  gtk.main()
'

[Martin Pool (mbp)]

Can you reproduce this in the tip of tribunal? I can't, for the moment.

Loading the file starves out other input, but it does complete fairly quickly.

[Martin Pool (mbp)]

With a change just committed to tip, Tribunal keeps the ui responsive while tests are loading so you can still look through things.

However, they still seem to load quite slowly compared to eg subunit-stats, so I need to make this faster.

[Martin Pool (mbp)]

Here's some sample data for the sake of reproducibility.

[Robert Collins (lifeless)]

Its about 5seconds per 1000 tests, judging from a quick eyeball.

[Robert Collins (lifeless)]

A little wrong, as I had to hit alt-f4 after it finished but:

robertc@lifeless-64:~/source/tribunal$ time subunit-stats < ~/source/baz/pending/working/test.log
There are 522 lines with trailing white space in 150 files.
There are 2041 lines longer than 79 characters in 337 files.
Total tests: 24616
Passed tests: 22383
Failed tests: 1
Skipped tests: 2232
Seen tags:

real 0m5.601s
user 0m5.530s
sys 0m0.050s
 robertc@lifeless-64:~/source/tribunal$ time (subunit-filter < ~/source/baz/pending/working/test.log | subunit-stats )
There are 522 lines with trailing white space in 150 files.
There are 2041 lines longer than 79 characters in 337 files.
Total tests: 2280
Passed tests: 47
Failed tests: 1
Skipped tests: 2232
Seen tags:

real 0m7.941s
user 0m10.930s
sys 0m0.230s

So, I'm pretty convinced there is something wrong here :)

testr can show it in 5 seconds too, FWIW, when set as .testrepository/failing

-Rob