Commit with non-ascii str committer value breaks with ElementTree 1.3

[Martin Packman]

On Python 2.7 the following tests (which are all the same test method) fail:

    bt.test_testament.TestamentTests.test_testament_unicode_commit_message
    bt.test_testament.TestamentTestsStrict.test_testament_unicode_commit_message
    bt.test_testament.TestamentTestsStrict2.test_testament_unicode_commit_message

The error is along the lines of:

    Traceback (most recent call last):
      File ".\bzrlib\tests\test_testament.py", line 133, in test_testament_unicode_commit_message
        revprops={'uni':u'\xb5'}
      File "<string>", line 4, in commit_write_locked
      File ".\bzrlib\workingtree_4.py", line 197, in commit
        result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
      File "<string>", line 4, in commit_write_locked
      File ".\bzrlib\mutabletree.py", line 200, in commit
        *args, **kwargs)
      File ".\bzrlib\commit.py", line 286, in commit
        possible_master_transports=possible_master_transports)
      File ".\bzrlib\cleanup.py", line 131, in run
        self.cleanups, self.func, self, *args, **kwargs)
      File ".\bzrlib\cleanup.py", line 165, in _do_with_cleanups
        result = func(*args, **kwargs)
      File ".\bzrlib\commit.py", line 430, in _commit
        self.rev_id = self.builder.commit(self.message)
      File ".\bzrlib\repository.py", line 190, in commit
        self.new_inventory, self._config)
      File ".\bzrlib\repository.py", line 1151, in add_revision
        self._add_revision(rev)
      File ".\bzrlib\repository.py", line 1154, in _add_revision
        text = self._serializer.write_revision_to_string(revision)
      File ".\bzrlib\xml_serializer.py", line 98, in write_revision_to_string
        return tostring(self._pack_revision(rev)) + '\n'
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 1120, in tostring
        ElementTree(element).write(file, encoding, method=method)
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 815, in write
        serialize(write, self._root, encoding, qnames, namespaces)
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 926, in _serialize_xml
        v = _escape_attrib(v, encoding)
      File ".\bzrlib\xml_serializer.py", line 147, in _escape_attrib
        elementtree.ElementTree._raise_serialization_error(text)
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 1046, in _raise_serialization_error
        "cannot serialize %r (type %s)" % (text, type(text).__name__)
    TypeError: cannot serialize 'Erik B\xe5gfors <test@user>' (type str)

If the monkey patching of escaping functions noted in bug 614522 is removed, the error ends instead:

    ...
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 926, in _serialize_xml
        v = _escape_attrib(v, encoding)
      File "...\python-trunk\lib\xml\etree\ElementTree.py", line 1084, in _escape_attrib
        return text.encode(encoding, "xmlcharrefreplace")
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 6: ordinal not in range(128)

Though this throws due to the update to ElementTree 1.3, the basic problem is the test passes a non-ascii str as the committer to the working tree commit method.

The tests can easily be fixed by the addition of a 'u' prefix, but shouldn't commit really be checking that it's given a value for committer it can use without things potentially breaking during serialisation?

[Martin Packman (gz)]

In absence of advice otherwise, I'll add a guard in commit that throws a UnicodeDecodeError on non-ascii byte strings rather than letting it through to the xml output layer.

[Martin Packman (gz)]

Added a guard in commit builder in the end.