bzr add * fails with non-ascii file names.

[Mario del Pozo]

Hello,

This bug report comes from a Windows XP SP2 English version but configured to Spanish locale.
Bazaar (bzr) 0.18.0

If you have a file with non-ascii characters on its file name, something like:

áéíóúÁÉÍÓÚñÑçC.txt

and you try to execute the add command like this:

bzr add *

fails with:

bzr arguments: [u'add', u'*']
looking for plugins in C:/Documents and Settings/Mario/Application Data/bazaar/2.0/plugins
looking for plugins in C:\WinApps\Dev\Bazaar\lib\library.zip\bzrlib\plugins
Looking for plugins in 'C:\\WinApps\\Dev\\Bazaar\\lib\\library.zip\\bzrlib\\plugins'
Names in archive: ['__init__.pyc', 'launchpad/__init__.pyc', 'launchpad/lp_indirect.pyc', 'launchpad/lp_registration.pyc', 'launchpad/test_lp_indirect.pyc', 'launchpad/test_register.pyc']
Load plugin launchpad from zip 'C:\\WinApps\\Dev\\Bazaar\\lib\\library.zip\\bzrlib\\plugins'
encoding stdout as sys.stdout encoding 'cp850'
Traceback (most recent call last):
  File "bzrlib\commands.pyc", line 729, in run_bzr_catch_errors
  File "bzrlib\commands.pyc", line 691, in run_bzr
  File "bzrlib\commands.pyc", line 389, in run_argv_aliases
  File "bzrlib\builtins.pyc", line 372, in run
  File "bzrlib\workingtree.pyc", line 337, in open_containing
  File "bzrlib\bzrdir.pyc", line 590, in open_containing
  File "bzrlib\lazy_import.pyc", line 116, in __call__
  File "bzrlib\transport\__init__.pyc", line 1127, in get_transport
  File "bzrlib\transport\__init__.pyc", line 1117, in convert_path_to_url
  File "bzrlib\urlutils.pyc", line 273, in _win32_local_path_to_url
  File "bzrlib\osutils.pyc", line 269, in _win32_abspath
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 0: ordinal not in range(128)

return code 3

But if you execute the add command like this, it works:

bzr add

or

bzr add .

or

bzr add áéíóúÁÉÍÓÚñÑçC.txt

bzr arguments: [u'add', u'\xe1\xe9\xed\xf3\xfa\xc1\xc9\xcd\xd3\xda\xf1\xd1\xe7C.txt']
looking for plugins in C:/Documents and Settings/Mario/Application Data/bazaar/2.0/plugins
looking for plugins in C:\WinApps\Dev\Bazaar\lib\library.zip\bzrlib\plugins
Looking for plugins in 'C:\\WinApps\\Dev\\Bazaar\\lib\\library.zip\\bzrlib\\plugins'
Names in archive: ['__init__.pyc', 'launchpad/__init__.pyc', 'launchpad/lp_indirect.pyc', 'launchpad/lp_registration.pyc', 'launchpad/test_lp_indirect.pyc', 'launchpad/test_register.pyc']
Load plugin launchpad from zip 'C:\\WinApps\\Dev\\Bazaar\\lib\\library.zip\\bzrlib\\plugins'
encoding stdout as sys.stdout encoding 'cp850'
opening working tree 'C:/_mario/Temp/bzr'
return code 0

Besr regards,
Mario

[Alexander Belchenko (bialix)]

Confirm this bug. bzr fails even with
bzr add *.txt

I think it's related to win32utils.glob_expand() function. It should return list of unicode filenames, but it seems that instead it returns plain strings. May be it's the bug in glob.glob()?

[Alexander Belchenko (bialix)]

It's indeed bug in glob.glob(). See:

C:\Temp\1>dir /b
.bzr
Тест.txt

C:\Temp\1>python -c "import glob; print glob.glob('*')"
['\xd2\xe5\xf1\xf2.txt']

C:\Temp\1>python -c "import glob; print glob.glob(u'*')"
['\xd2\xe5\xf1\xf2.txt']

Running glob.glob() either with plain string pattern, or with unicode one, I always get back list of plain strings.
I think it's serious bug. It's not critical, but serious enough.

[Kuno Meyer (kuno-meyer)]

This is a known bug of glob.glob():

"[ 1001604 ] glob doesn't return unicode with no dir in unicode filename"
https://sourceforge.net/tracker/index.php?func=detail&aid=1001604&group_id=5470&atid=305470

[Alexander Belchenko (bialix)]

Kuno wrote bugfix and corresponding tests.

[Alexander Belchenko (bialix)]

merged in bzr.dev as revno.2655