can't push or pull unicode names into or out of an svn checkout

[Wesley J. Landaker]

Normally, pushing and pulling (or checking out) from an svn repository or an svn checkout works fine. However, when Unicode filenames are involved, it fails when targeting an svn checkout. (Note that the same thing applies to bzr-svn lightweight checkouts, since they are just really just svn checkouts.)

I've reported a number of Unicode-related bugs in the past. I don't believe this is a duplicate of any of them.

With bzr 1.9 and bzr-svn 0.4.15 (both from Ubuntu intrepid packages):

First, setup a temporary environment:
mkdir /tmp/bzr-svn-test
cd /tmp/bzr-svn-test
svnadmin create svn-repos
svn co file://`pwd`/svn-repos svn-co
bzr branch svn-repos bzr-branch

Then, try to add a file named "✼" from bzr and push it into the repository:
cd bzr-branch
touch ✼
bzr add
bzr commit -m "Added filename with Unicode U+273C"
bzr push ../svn-co

This results in:
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 913, in run
    use_existing_dir=use_existing_dir)
  File "/usr/lib/python2.5/site-packages/bzrlib/push.py", line 168, in _show_push_branch
    tree_to.update()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 176, in update
    self.read_working_inventory()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 371, in read_working_inventory
    add_dir_to_inv(u"", rootwc, None)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 335, in add_dir_to_inv
    (id, revid) = find_ids(entry, rootwc)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 312, in find_ids
    relpath)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 254, in path_to_file_id
    entry = self.basis_tree().id_map[rp.decode("utf-8")]
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/workingtree.py", line 578, in basis_tree
    self.base_tree = SvnBasisTree(self)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/tree.py", line 231, in __init__
    workingtree.branch.mapping)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/repository.py", line 400, in get_fileid_map
    return self.fileid_map.get_map(self.uuid, revnum, path, mapping)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/fileids.py", line 294, in get_map
    self.actual.update_map(map, revid, idmap, changes)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/fileids.py", line 193, in update_map
    for x in sorted(delta.keys(), reverse=True):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

bzr 1.9 on python 2.5.2 (linux2)
arguments: ['/usr/bin/bzr', 'push', '../svn-co/']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bisect /home/wjlanda/.bazaar/plugins/bisect [1.1.0pre0]
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.9.1]
  cvsps_import /usr/lib/python2.5/site-packages/bzrlib/plugins/cvsps_import [unknown]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.96.0dev1]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  loom /usr/lib/python2.5/site-packages/bzrlib/plugins/loom [1.4.0dev0]
  rebase /usr/lib/python2.5/site-packages/bzrlib/plugins/rebase [0.4.1]
  search /usr/lib/python2.5/site-packages/bzrlib/plugins/search [1.6.0dev3]
  stats /usr/lib/python2.5/site-packages/bzrlib/plugins/stats [unknown]
  svn /usr/lib/python2.5/site-packages/bzrlib/plugins/svn [0.4.15dev0]
  upload /usr/lib/python2.5/site-packages/bzrlib/plugins/upload [0.1.0]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

Instead if you push to the respository it works:
bzr push ../svn-repos

This results in:
Committing to: /tmp/bzr-svn-test/bzr-branch/
added ✼
Committed revision 2.

Now trying to check it out works:
cd ..
bzr branch svn-repos bzr-branch2
bzr checkout svn-repos bzr-co-heavy

But trying to branch or checkout from a checkout fails, or trying to make a lightweight checkout both fail:
bzr branch svn-co bzr-branch3
bzr checkout svn-co bzr-branch3
bzr checkout --lightweight svn-repos bzr-co-light

When they fail, they all give:

bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

and a similar backtrace.

[Jelmer Vernooij (jelmer)]

I can reproduce this in 0.4, but it's fixed in 0.5.