addDetail does not work with binary content in python3

[Trevor McCasland]

As a user of the testtools library, I would like to support adding details to test cases in binary format for python3. The following is a scenario to reproduce the bug. As a result of this error, the details of the test are excluded from the subunit streams generated in python3. I can provide more details on the subunit stream if that is desired.

================================
Test case used to add attachment
================================
# note here I have to use utf8 encoding for python3
# https://stackoverflow.com/questions/29751751/string-to-bytes-in-both-python-2-and-3

class TestReadSubunit(base.TestCase):
    def test_get_duration(self):
        dur = subunit.get_duration(datetime.datetime(1914, 6, 28, 10, 45, 0),
                                   datetime.datetime(1914, 6, 28, 10, 45, 50))
        self.assertEqual(dur, 50.000000)
        ctnt_type = content_type.ContentType('application', 'octet-stream')
        if sys.version_info < (3, 0):
            data = bytes('data')
        else:
            data = bytes('data', 'utf8')
        ctnt = content.Content(ctnt_type, lambda: data)
        self.addDetail('test binary content', ctnt)

# adding binary content in python3 causes an error in the __repr__ call and self.getDetails() call

(py35) ubuntu@ubuntu:~/subunit2sql$ python -m testtools.run subunit2sql.tests.test_read_subunit.TestReadSubunit.test_get_duration
Tests running...
> /home/ubuntu/subunit2sql/subunit2sql/tests/test_read_subunit.py(41)test_get_duration()
-> self.addDetail('test binary content', ctnt)
(Pdb) ctnt
*** TypeError: sequence item 0: expected a bytes-like object, int found
(Pdb) self.getDetails()
*** TypeError: sequence item 0: expected a bytes-like object, int found

# no problem in python2

(py27) ubuntu@ubuntu:~/subunit2sql$ python -m testtools.run subunit2sql.tests.test_read_subunit.TestReadSubunit.test_get_duration
Tests running...
> /home/ubuntu/subunit2sql/subunit2sql/tests/test_read_subunit.py(41)test_get_duration()
-> self.addDetail('test binary content', ctnt)
(Pdb) ctnt
<Content type=application/octet-stream, value='data'>
(Pdb) self.getDetails()
{'test binary content': <Content type=application/octet-stream, value='data'>}

[Trevor McCasland (twm2016)]

This is more likely a bug with the Content object not being able to join a binary string together rather than the addDetail method "working" correctly.

[Robert Collins (lifeless)]

I think you're mis-using the API, at least in the example given. You can see here - https://github.com/testing-cabal/testtools/blob/master/testtools/content.py#L94 - that _get_bytes should return an iterable of bytestrings.

In Python2, b'123' is really just a 'str', and for str's, iter('123') -> ['1', '2', '3'].
In Python3, b'123' is a distinct type, and iter(b'123') -> [ord('1'), ord('2'), ord('3')].

So the minimal change to fix your example would be to change lambda: data to lambda:[data].

FWIW you can use b'' in Python 2.7 and any supported version 3, making byte literals super easy: the stack exchange post you link to is only relevant if you don't know the type of the data before it reaches the function. That ambiguity is common in Python2 but extremely rare in 3 - we've found that once a code base is converted to 3 the ambiguity pretty much disappears even in Python2 (at least once stdin is setup appropriately).

All of that said, I think using the helpers we have should meet 99% of needs:
literal text: content.text_content
json: content.json_content
files: content.attach_file(self, 'path/to/logfile', content_type=content_type.ContentType('...', '...')
sockets and other iterable objects: content.content_from_stream(f)
not iterable, but readable: content.content_from_reader

tl;dr: if you have a literal string, use content.text_content('my string'). If you have actual bytes (e.g. a log file or whatever),

Hope this helps; I'm closing this as I'm very sure of the reason for the stacktrace, but please do re-open if I'm wrong ;).