testr reports double failure counts

Bug #1331140 reported by Dan Smith on 2014-06-17
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Testrepository
Undecided
Unassigned

Bug Description

When I run testr as part of nova's unit tests, any failures get reported twice in the count. The actual failure information is only reported once, but if two tests fail, it reports four failures at the end. This happens all the time as best I can tell.

Russell Bryant (russellb) wrote :

Can confirm that this happens to me as well with nova unit tests

Michael H Wilson (geekinutah) wrote :

I also can confirm

Dan Smith (danms) on 2014-06-17
Changed in testrepository:
status: New → Confirmed
Robert Collins (lifeless) wrote :

I can't confirm this - can I see a trace of the problem please? e.g. just the output from a run where one test fails but you see two counted as failing.

Changed in testrepository:
status: Confirmed → Incomplete
Russell Bryant (russellb) wrote :
Download full text (3.7 KiB)

I can't reproduce the bug anymore, either. If it still exists, it's not as straight forward as I described it in this report. I think you can close this one. Thanks for looking!

$ printf "\n def test_fail(self):\n raise Exception()" >> nova/tests/test_utils.py
$ tox -e py27 nova.tests.test_utils
py27 develop-inst-noop: /home/rbryant/src/nova
py27 runtests: PYTHONHASHSEED='0'
py27 runtests: commands[0] | python -m nova.openstack.common.lockutils python setup.py test --slowest --testr-args=nova.tests.test_utils
running test
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpn0fbX6
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpNoCEdP
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmp9q4dRw
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpYA_RfD
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpqtYqPO
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpTftW8h
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpluy0GT
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --load-list /tmp/tmpeyIcum
======================================================================
FAIL: nova.tests.test_utils.ConstantTimeCompareTestCase.test_fail
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "nova/tests/test_utils.py", line 987, in test_fail
    raise Exception()
Exception
Traceback (most recent call last):
_StringException: Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "nova/tests/test_utils.py", line 987, in test_fail
    raise Exception()
Exception

Traceback (most recent call last):
_StringException...

Read more...

Changed in testrepository:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers