bug failure breaks our subunit output

[Gary Poster]

As found in http://ubuntuone.com/1J4THLUkQKfWMlpqvQgDmm , we had a test failure that broke our subunit output somehow. subunit was unable to recognize any tests from the stream after the failing test lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships (http://pastebin.ubuntu.com/1153299/) . I don't yet clearly see what is broken in that subunit output, but whatever it is should be addressed. If we can also make subunit more robust, that's a bonus, but that should be a separate task/bug.

[Robert Collins (lifeless)]

test: lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships
time: 2012-08-16 19:25:00.028197Z
tags: -zope:layer:lp.testing.layers.LaunchpadScriptLayer zope:layer:lp.testing.layers.BaseLayer -zope:layer:lp.codehosting.codeimport.tests.test_worker.ForeignBranchPluginLayer zope:layer:lp.testing.layers.LaunchpadFunctionalLayer zope:layer:lp.testing.layers.DatabaseLayer zope:layer:lp.testing.layers.RabbitMQLayer -zope:layer:lp.testing.layers.ZopelessLayer zope:layer:lp.testing.layers.LibrarianLayer -zope:layer:lp.codehosting.tests.test_acceptance.SSHServerLayer -zope:layer:lp.testing.layers.ZopelessAppServerLayer zope:layer:lp.testing.layers.MemcachedLayer zope:layer:lp.testing.layers.FunctionalLayer -zope:layer:lp.testing.layers.LaunchpadZopelessLayer zope:layer:lp.testing.layers.LaunchpadLayer
tags: worker-13
failure: lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships [ multipart

Goes
test: A
time:
tags:
tags:
failure: B

That A and B need to match - subunit thinks its still waiting on an outcome for A. I see no further references to A in the stream.

[Benji York (benji)]

Robert Collins (lifeless) wrote:
> test: lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships
> [...]
> failure: lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships [ multipart
>
> Goes
> test: A
> time:
> tags:
> tags:
> failure: B

I don't follow. Both the "test:" and "failure:" lines reference
"lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships".

[Brad Crittenden (bac)]

We have been unable to reproduce the problem using subunit filtering of the problem file. Marking the bug as incomplete.

[Robert Collins (lifeless)]

 Bah, this is why you don't do protocol debugging at 3:30am, no matter how helpful you want to be.

[Robert Collins (lifeless)]

Indeed, the attached stream looks fine to me, and subunit-stats sees:
Total tests: 21791
Passed tests: 21260
subunit-ls reports tests after the named one:
lp.bugs.browser.tests.test_bugtask.TestBugTaskView.test_rendered_query_counts_constant_with_team_memberships
lp.testing.layers.ZopelessAppServerLayer:setUp
...

[Launchpad Janitor (janitor)]

[Expired for Launchpad itself because there has been no activity for 60 days.]