One error is reported as two in the totals

[Nate Johnston]

Sometimes when an error is detected, that single error is reported, but in the "Totals" section the number of errors is counted as two.

Example:

http://logs.openstack.org/24/299024/2/check/gate-neutron-dsvm-fullstack/74fede1/console.html#_2016-04-11_06_01_38_876

2016-04-11 06:01:38.845 | 2016-04-11 06:01:38.832 | ==============================
2016-04-11 06:01:38.852 | 2016-04-11 06:01:38.839 | Failed 1 tests - output below:
2016-04-11 06:01:38.854 | 2016-04-11 06:01:38.842 | ==============================
...
2016-04-11 06:01:40.502 | 2016-04-11 06:01:40.489 | ======
2016-04-11 06:01:40.503 | 2016-04-11 06:01:40.491 | Totals
2016-04-11 06:01:40.507 | 2016-04-11 06:01:40.495 | ======
2016-04-11 06:01:40.512 | 2016-04-11 06:01:40.500 | Ran: 14 tests in 237.0000 sec.
2016-04-11 06:01:40.515 | 2016-04-11 06:01:40.503 | - Passed: 11
2016-04-11 06:01:40.516 | 2016-04-11 06:01:40.504 | - Skipped: 1
2016-04-11 06:01:40.518 | 2016-04-11 06:01:40.506 | - Expected Fail: 0
2016-04-11 06:01:40.534 | 2016-04-11 06:01:40.509 | - Unexpected Success: 0
2016-04-11 06:01:40.534 | 2016-04-11 06:01:40.511 | - Failed: 2
2016-04-11 06:01:40.534 | 2016-04-11 06:01:40.516 | Sum of execute time for each test: 1517.0934 sec.

I think os-testr is the right place to put this bug, but let me know if it needs to be elsewhere. Thanks!

[Matthew Treinish (treinish)]

This is actually an underlying issue with testr that has been so longstanding that ostestr tries to work around it. Testr reports 2 failures 1 of which is an internal artifact and subunit-trace (which is the output filter being used by ostestr) is filtering that from the test results:

https://github.com/openstack/os-testr/blob/master/os_testr/subunit_trace.py#L170-L172

However, the count generation just looks at the raw result output and counts test states (and since there are 2 failures in the output, without any other context) the count shows both.

https://github.com/openstack/os-testr/blob/master/os_testr/subunit_trace.py#L252-L259