testr reports wrong work # on tests some times

Bug #1316858 reported by Sean Dague on 2014-05-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Testrepository
High
Robert Collins

Bug Description

Using the new subunit-trace.py filter in tempest we've discovered that some times the worker # for a subunit stream is incorrect. On a 4 worker environment we can often get 1, 2, 3, 3 (noting no 0) as reported workers for the tests. There is an attached tempest partial run with subunit test lists, and subunit streams demonstrating this.

Sean Dague (sdague) wrote :
Matt Riedemann (mriedem) wrote :

This shows up a lot: http://goo.gl/IZi6pO

Changed in testrepository:
status: New → Confirmed
Changed in testrepository:
importance: Undecided → High
status: Confirmed → Triaged
Robert Collins (lifeless) wrote :

Hi, thanks for filing this. The trace you got is great, but its missing what testrepository saved to disk, so I can't see whats wrong :(

Robert Collins (lifeless) wrote :

To reproduce, in the testrepository tree itself run

./testr run --parallel --subunit | ~/work/openstack/tempest/tools/subunit-trace.py

 - WARNING: missing Worker 0! Race in testr accounting.
 - Worker 1 (170 tests) => 0:00:01.895823s
 - WARNING: missing Worker 2! Race in testr accounting.
 - Worker 3 (174 tests) => 0:00:01.917335s

Robert Collins (lifeless) wrote :

I am thinking this is a testtools issue -

greedy evaluating the tests iterator in ConcurrentStreamTestSuite triggers

 - WARNING: missing Worker 0! Race in testr accounting.
 - WARNING: missing Worker 1! Race in testr accounting.
 - WARNING: missing Worker 2! Race in testr accounting.
 - Worker 3 (344 tests) => 0:00:01.939864s

reliably - 100% of the time

Robert Collins (lifeless) wrote :

Ok, StreamTagger is being constructed with bad data by the make tests lambda:

        super(StreamTagger, self).__init__(targets)
        self.add = frozenset(add or ())
        self.discard = frozenset(discard or ())
        print self.add

frozenset(['worker-3'])
frozenset(['worker-3'])
frozenset(['worker-3'])
frozenset(['worker-3'])

Changed in testrepository:
assignee: nobody → Robert Collins (lifeless)
milestone: none → next
Changed in testrepository:
status: Triaged → Fix Committed
Changed in testrepository:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers