UnicodeDecodeError in walkdirs called from initialize, on badly encoded filename

[NeilGreenwood]

I don't see why we should be calling build_tree at all for initialize; and if there is a bad filename we should say what specifically it is.

---

Binary package hint: bzr

neil@iodine:~$ bzr init
No handlers could be found for logger "bzr"
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xbf in position 19: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 716, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 911, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 547, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1564, in run
    possible_transports=[to_transport])
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 490, in create_branch_convenience
    bzrdir.create_workingtree()
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1481, in create_workingtree
    accelerator_tree=accelerator_tree, hardlink=hardlink)
  File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree_4.py", line 1394, in initialize
    hardlink=hardlink, delta_from_tree=True)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line 2034, in build_tree
    delta_from_tree)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line 2050, in _build_tree
    for dir, files in wt.walkdirs():
  File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree.py", line 2372, in walkdirs
    current_disk = disk_iterator.next()
  File "/usr/lib/python2.6/dist-packages/bzrlib/osutils.py", line 1308, in walkdirs
    names = sorted(_listdir(top))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbf in position 19: ordinal not in range(128)

bzr 1.13.1 on python 2.6.2 (linux2)
arguments: ['/usr/bin/bzr', 'init']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_GB.UTF-8'
plugins:
  bzrtools /usr/lib/python2.6/dist-packages/bzrlib/plugins/bzrtools [1.13]
  etckeeper /usr/lib/python2.6/dist-packages/bzrlib/plugins/etckeeper [unknown]
  launchpad /usr/lib/python2.6/dist-packages/bzrlib/plugins/launchpad [unknown]
  netrc_credential_store /usr/lib/python2.6/dist-packages/bzrlib/plugins/netrc_credential_store [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

neil@iodine:~$ lsb_release -rd
Description: Ubuntu 9.04
Release: 9.04

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
Package: bzr 1.13.1-1
ProcEnviron:
 PATH=(custom, user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: bzr
Uname: Linux 2.6.28-15-generic i686

[Robert Collins (lifeless)]

@Alex, please do not mark bugs as duplicates that are not duplicates. If you're not sure *do not dup them*.

@Neil what does 'pwd' show for you?

It looks like you have a UTF8 environment but a non-utf8 path element in your current working directory.

[NeilGreenwood (neil-greenwood)]

neil@iodine:~$ pwd
/home/neil
neil@iodine:~$ pwd -L
/home/neil
neil@iodine:~$ pwd -P
/home/neil

neil@iodine:~$ locale
LANG=en_GB.UTF-8
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=

You're right about the environment. I guess it's a subdirectory (that bzr was recursing through) rather than my current path. Any suggestions about how I could find the culprit?

[Robert Collins (lifeless)]

Ah, are you trying to init your entire home dir, or a new project under
it?

If the former, try this:
mkdir temp
cd temp
bzr init
bzr mv .bzr ../
cd ..
bzr st
hopefully that will blow up with a more useful error.

If the latter, then its a sibling - something in your homedir, I guess.

-Rob

[NeilGreenwood (neil-greenwood)]

Yes, it's the entire home dir I'm trying to init (well, actually the dot-files in it).

neil@iodine:~$ mkdir temp
neil@iodine:~$ cd temp
neil@iodine:~/temp$ bzr init
No handlers could be found for logger "bzr"
Created a standalone tree (format: pack-0.92)
neil@iodine:~/temp$ bzr mv .bzr ../
No handlers could be found for logger "bzr"
bzr: ERROR: ../ is not in the same branch as .bzr

That error might be because I have a .bzr directory in my home directory. 'bzr st' in my home directory lists every file/dir as status Unknown.

[Robert Collins (lifeless)]

On Mon, 2009-09-07 at 21:03 +0000, NeilGreenwood wrote:
> neil@iodine:~/temp$ bzr mv .bzr ../
> No handlers could be found for logger "bzr"
> bzr: ERROR: ../ is not in the same branch as .bzr
>
> That error might be because I have a .bzr directory in my home
> directory. 'bzr st' in my home directory lists every file/dir as
> status
> Unknown.

Actually, you did 'bzr mv .bzr ../' rather than 'mv .bzr ../' :)

So you have a .bzr in ~ already - how come you need to 'init' it then?

What does 'ls -l ~/.bzr' show?

-Rob

[Martin Pool (mbp)]

Alex, you're welcome to say "maybe this is a dupe of bug 400669" if
you think it is, but you are not sure the stacktrace or cause is the
same.

--
Martin <http://launchpad.net/~mbp/>

[NeilGreenwood (neil-greenwood)]

@Robert,

I did "bzr mv .bzr .." because that's what you told me to type! I did wonder if I should just move it. :-)

Not sure if the failed command created the ~/.bzr directory or not. I'll check the creation dates on it and examine its contents when I get home.

[NeilGreenwood (neil-greenwood)]

neil@iodine:~$ ls -ld .bzr
drwxr-xr-x 6 neil neil 4096 2009-09-06 21:40 .bzr
neil@iodine:~$ ls -l .bzr
total 24
drwxr-xr-x 3 neil neil 4096 2009-09-06 21:40 branch
-rw-r--r-- 1 neil neil 35 2009-09-06 21:40 branch-format
drwxr-xr-x 2 neil neil 4096 2009-09-06 21:40 branch-lock
drwxr-xr-x 3 neil neil 4096 2009-09-06 21:40 checkout
-rw-r--r-- 1 neil neil 141 2009-09-06 21:40 README
drwxr-xr-x 7 neil neil 4096 2009-09-06 21:40 repository
neil@iodine:~$

I didn't check before I tried the 'bzr mv .bzr ..' - would that have (partially) overwritten an existing directory? I moved the .bzr directory, then tried the 'bzr init' again in my home directory. Got the same error/stacktrace as originally reported, and this is the contents of the ~/.bzr directory that results:
neil@iodine:~$ ls -l .bzr
total 24
drwxr-xr-x 3 neil neil 4096 2009-09-09 20:41 branch
-rw-r--r-- 1 neil neil 35 2009-09-09 20:41 branch-format
drwxr-xr-x 2 neil neil 4096 2009-09-09 20:41 branch-lock
drwxr-xr-x 3 neil neil 4096 2009-09-09 20:41 checkout
-rw-r--r-- 1 neil neil 141 2009-09-09 20:41 README
drwxr-xr-x 7 neil neil 4096 2009-09-09 20:41 repository
neil@iodine:~$

bzr st still shows all files/directories in an unknown state.

[NeilGreenwood (neil-greenwood)]

Ah! I've just realised that everything being in the unknown state means I can add them into version control as I wish.

At the very least though, there should be a better error message from bzr when this situation occurs, including a better identification of the filename that's causing the problem.