testtools-0.9.14 outplays subunit

[IAN DELANEY]

Testing subunit-0.0.7, it fails to start the test suite. The testtools version 0.9.14 has made some change that renders it incomaptible.

archtester subunit # ebuild subunit-0.0.7.ebuild clean test

libtool: link: x86_64-pc-linux-gnu-gcc -I./c/include -march=native -pipe -O2 -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -o c/tests/.libs/test_child c/tests/c_tests_test_child-test_child.o -lcheck ./.libs/libsubunit.so
make[1]: Nothing to be done for `runtests.py'.
make[1]: Leaving directory `/mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.7/work/subunit-0.0.7-2.7'
make check-TESTS check-local
make[1]: Entering directory `/mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.7/work/subunit-0.0.7-2.7'
Traceback (most recent call last):
  File "./runtests.py", line 23, in <module>
    from subunit.tests.TestUtil import TestVisitor, TestSuite
  File "/mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.7/work/subunit-0.0.7-2.7/python/subunit/tests/__init__.py", line 17, in <module>
    from subunit.tests import (
  File "/mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.7/work/subunit-0.0.7-2.7/python/subunit/tests/test_test_protocol.py", line 25, in <module>
    from testtools.tests.helpers import (
ImportError: cannot import name Python26TestResult
FAIL: runtests.py
Running suite(s): subunit_child
100%: Checks: 6, Failures: 0, Errors: 0
PASS: c/tests/test_child
================================================
1 of 2 tests failed
Please report to <email address hidden>
================================================
make[1]: *** [check-TESTS] Error 1
make[1]: Leaving directory `/mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.7/work/subunit-0.0.7-2.7'
make: *** [check-am] Error 2
 * ERROR: dev-python/subunit-0.0.7 failed (test phase):
 * Testing failed with CPython 2.7 in python_default_function() function
 *

I'm still perplexed by the function name Python26TestResult. Initially it seemed it was seeking a non-existing function. The test suite works fine with testtools-0.9.11. The line is the same in subunit-0.0.7-2.7/python/subunit/tests/test_test_protocol.py.

[Jonathan Lange (jml)]

Ugh. Sorry. I don't know how this happened. We really need some integration bots that run released subunit against testtools. Static analysis reveals that the problem was introduced in testtools 0.9.12.

In any case, subunit trunk has this problem fixed. We're unlikely to do a point release for testtools 0.9.12, and I'd rather not restore the compatibility kludge. Would it be OK if we pushed for a new release of subunit instead?

[Arfrever Frehtes Taifersar Arahesis (arfrever-fta)]

Please fix bug #987490 and bug #987514 before releasing new version of Subunit.

[Robert Collins (lifeless)]

I've checked in subunit, and while testtools could avoid the import error by doing:
=== modified file 'testtools/tests/helpers.py'
--- testtools/tests/helpers.py 2012-02-04 16:47:09 +0000
+++ testtools/tests/helpers.py 2012-05-07 18:58:57 +0000
@@ -18,6 +18,7 @@

 # Importing to preserve compatibility.
 safe_hasattr
+from testtools.testresult.doubles import Python26TestResult, Python27TestResult, ExtendedTestResult

 # GZ 2010-08-12: Don't do this, pointlessly creates an exc_info cycle
 try:

that would not be enough to keep subunit 0.0.7's test suite working - it uses the result of stringification in its tests:
(<subunit.RemotedTestCase description='bing crosby'>, '_StringException: foo.c:53:ERROR invalid state\n\n')
(<subunit.RemotedTestCase description='bing crosby'>, '_StringException: Text attachment: traceback\n------------\nfoo.c:53:ERROR invalid state\n------------\n\n')

So we can't change this unless we have an option or something, and force folk to opt in.

That said, subunit 0.0.7 works even though the test suite fails.

So I'm going to wontfix this particular bug: but I'm on a warpath to releases subunit 0.0.8 today, with the python 3 patch thats in the bug tracker, and it and testtools 0.9.15 should play together real nice.

[IAN DELANEY (johneed)]

good one. thanks for follow up. We'' try it out soon.

[IAN DELANEY (johneed)]

I am kind of confused. I got the tests suite to pass in python2.6 at last.
Correct me if I'm wrong here.

http://bazaar.launchpad.net/~subunit/subunit/trunk/files/head:/python/subunit/

lists

tests
__init__.py
chunked.py
details.py
filters.py
iso8601.py
progress_model.py
run.py
test_results.py

archtester subunit # ls /mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.8/work/subunit-0.0.8/python/subunit/
chunked.py details.py __init__.py iso8601.py progress_model.py run.py test_results.py tests

filters.py is missing.

http://bazaar.launchpad.net/~subunit/subunit/trunk/files/head:/python/subunit/tests/

lists

__init__.py sample-script.py sample-two-script.py
test_chunked.py test_details.py test_progress_model.py
tets_run.py test_subunit_filter.py test_subunit_stats.py
test_subunit_tags.py test_tap2subunit.py test_test_protocol.py
test_test_results.py TestUtil.py

and

archtester subunit # ls /mnt/gen2/TmpDir/portage/dev-python/subunit-0.0.8/work/subunit-0.0.8/python/subunit/tests

__init__.py test_chunked.py test_subunit_filter.py test_tap2subunit.py TestUtil.py
sample-script.py test_details.py test_subunit_stats.py test_test_protocol.py
sample-two-script.py test_progress_model.py test_subunit_tags.py test_test_results.py

test_run.py is missing.

Adding them;

archtester subunit # ls files
filters.py test_run.py

yields

----------------------------------------------------------------------
Ran 266 tests in 0.778s

OK
PASS: runtests.py
Running suite(s): subunit_child
100%: Checks: 6, Failures: 0, Errors: 0
PASS: c/tests/test_child
==================
All 2 tests passed
==================

well anyway, once added, the test suite passes with ALL python ABIs.