UnicodeError writing change summary into commit message template

[James Henstridge]

I got the following error when trying to commit after a merge changes from the Launchpad rocketfuel branch.

$ bzr commit
bzr: ERROR: 'ascii' codec can't encode character u'\ufffd' in position 29: ordinal not in range(128)
  command: '/usr/bin/bzr' 'commit'
      pwd: u'/home/james/src/launchpad'
    error: exceptions.UnicodeEncodeError
  at /usr/lib/python2.4/site-packages/bzrlib/status.py line 108, in show_status()
  see ~/.bzr.log for debug information

The relevant entry from ~/.bzr.log is:
[10834] Wed 14:57:08.264 ERROR: 'ascii' codec can't encode character u'\ufffd' in position 29: ordinal not in range(128)
  command: '/usr/bin/bzr' 'commit'
      pwd: u'/home/james/src/launchpad'
    error: exceptions.UnicodeEncodeError
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 522, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 497, in run_bzr
    ret = cmd_obj.run_argv(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 215, in run_argv
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.4/site-packages/bzrlib/builtins.py", line 1068, in run
    to_file=catcher)
  File "/usr/lib/python2.4/site-packages/bzrlib/status.py", line 108, in show_status
    print >> to_file, ' ', line_log(mm_revision, 75)
UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 29: ordinal not in range(128)

This is with jbailey's bzr packages (bzr-0.1.1+20051026-0 and bzrtools-0.7+20051102-0).

[Martin Pool (mbp)]

The problem is caused by changeset Arch-1:<email address hidden>%launchpad--pre-ubz-specs--0--patch-22

This contains a \xfffd, which is a unicode replacement character. That character can't be converted to ascii.

This character occurs in the changeset because the original arch changeset contains a \xc2 character (meta-b), probably because of a typo when it was originally committed. This is not a valid utf-8 character (or unicode character.) I don't know what character set baz import expects the arch log messages to be; it's not specified by baz.

So a couple of possible changes:

 - when producing the commit message template we should write out in the user's preferred encoding

 - that encoding should be done with errors=replace so that users who are using ascii won't be messed up by strange things in merged commits

 - baz import should perhaps be smarter about non-ascii arch messages

James can work around this by giving the commit message from a file or on the command line, avoiding generation of the template.

[Jan Hudec (bulb)]

Actually happens with bzr.dev itself, since the name 'Erik Bågfors' can't be converted to ascii. The backtrace seems to be slightly different here (with just updated devel bazaar):

[16357] Sun 16:25:11.486 ERROR: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 25394: ordinal not in range(128)
  at /home/bulb/dev/bzr/bzr.dev/bzrlib/msgeditor.py line 95
  in edit_commit_message
Traceback (most recent call last):
  File "/home/bulb/dev/bzr/bzr.dev/bzrlib/commands.py", line 650, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/home/bulb/dev/bzr/bzr.dev/bzrlib/commands.py", line 612, in run_bzr
    ret = run(*run_argv)
  File "/home/bulb/dev/bzr/bzr.dev/bzrlib/commands.py", line 246, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/bulb/dev/bzr/bzr.dev/bzrlib/builtins.py", line 1553, in run
    message = edit_commit_message(template)
  File "/home/bulb/dev/bzr/bzr.dev/bzrlib/msgeditor.py", line 95, in edit_commit_message
    msgfile.write("\n\n%s\n\n%s" % (ignoreline, infotext))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 25394: ordinal not in range(128)

[John A Meinel (jameinel)]

This should be fixed in bzr.dev, if you find it is not, please re-open the bug.