assertThat style gives overly verbose output
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
testtools |
Fix Released
|
Wishlist
|
Jonathan Lange |
Bug Description
Currently, assertThat seems like a usability regression. To demonstrate, compare the same test from wikkid failing twice, one on a testtools subclass where assertThat is used, and one also subclassing the Bazaar case class which uses their assertEquals method rather than the testtools implementation:
Tests running...
=======
FAIL: wikkid.
-------
Text attachment: traceback
------------
Traceback (most recent call last):
File "C:\bzr\
return fn(*args)
File "C:\bzr\
return self._get_
File "C:\bzr\
self.
File "C:\bzr\
self.
File "C:\bzr\
self.
AssertionError: Match failed. Matchee: "None"
Matcher: Equals(
Difference: 'text/x-c++src' != None
------------
=======
FAIL: wikkid.
-------
Traceback (most recent call last):
File "C:\bzr\
self.
AssertionError: not equal:
a = 'text/x-c++src'
b = None
Ran 114 tests in 5.078s
FAILED (failures=2)
Robert noted two main differences:
# The extra levels of traceback, which he finds useful as he works on the test code itself. I'd suggest most people don't, and only care about their tests. The <lp:~gz/testtools/cleaner_tracebacks> branch would fix this, but makes life harder for the Roberts of the world and there doesn't seem much interest in landing it.
# The formatting is different. Robert felt there isn't much different between them, but the assertThat version prints the expected and actual output twice and uses longer more complicated words. It should make it clearer which is the expected value however, if the test is correctly written. Also, exarkun noted he liked being able to paste the test output into a repl which the Bazaar version enables.
See also bug 660852 as a wider output verbosity issue.
Related branches
- Robert Collins: Approve
- Michael Hudson-Doyle (community): Approve
-
Diff: 148 lines (+37/-33)3 files modifiedNEWS (+4/-0)
testtools/testcase.py (+9/-4)
testtools/tests/test_testcase.py (+24/-29)
Changed in testtools: | |
status: | New → Triaged |
importance: | Undecided → Low |
Changed in testtools: | |
importance: | Low → Wishlist |
tags: | added: matchers |
Changed in testtools: | |
status: | Triaged → In Progress |
assignee: | nobody → Jonathan Lange (jml) |
Changed in testtools: | |
status: | In Progress → Fix Committed |
milestone: | none → next |
Changed in testtools: | |
status: | Fix Committed → Fix Released |
It's not quite comparing apples with apples, since the Equals mismatch object special-cases short inequalities.
I personally think that the default should be to hide the layers of stack that don't matter for most test authors, and that we should provide some way to make those layers more visible for test framework folk. IIRC, recent versions of unittest have some built in support for this.