UnicodeDecodeError in get_password if user name is unicode

[Alexey Gaynulin]

This bug was described in post:
https://bugs.launchpad.net/bzr/+bug/140563
(the problem is stipulated by the "Alexey Gaynulin")

bzr init-repository --no-trees sftp://bazaar.09248.ru/bzr

Send three files which describes the logs and the errors.

When work with server: try to push, add, upgrade changes to the server cause errors of almost the same content:
 bzr: ERROR: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u0438' in position 0: ordinal not in range(128)

All the neccessary info is attached.
Please help (try to start work with bazaar but have't started yet)

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> ** Attachment added: "the logs and errors' report"
> http://launchpadlibrarian.net/50063597/UnicodeError%20-%20Alex%20Gaynulin%20-%20stdout%20stderr%2C%20logs%20and%20error%20reports.zip

Alexey, please, next time, attach them as separate files (you can attach
more than one file to bug report) instead of zipping them. Thanks.

[Alexander Belchenko (bialix)]

OK, as I understand this problem occurs because you have not set your
identity.

Please, run command `bzr whoami` and show us the output.

[Alexander Belchenko (bialix)]

The real problem is password prompt:

0.044 bazaar version: 2.1.1
0.044 bzr arguments: [u'push', u'sftp://bazaar.09248.ru/bzr']
0.058 looking for plugins in C:/Users/Алексей/AppData/Roaming/bazaar/2.0/plugins
0.058 looking for plugins in C:/Program Files/Bazaar/plugins
0.182 encoding stdout as sys.stdout encoding 'cp866'
0.217 opening working tree 'C:/Users/\xd0\x90\xd0\xbb\xd0\xb5\xd0\xba\xd1\x81\xd0\xb5\xd0\xb9/Documents/bazar/bzr'
0.278 bzr-svn: using Subversion 1.6.6 ()
0.307 falling back to default implementation
0.308 failed to load system host keys: [Errno 2] No such file or directory: 'C:\\Users\\\xc0\xeb\xe5\xea\xf1\xe5\xe9/.ssh/known_hosts'
[ 3336] 2010-06-10 12:38:30.688 INFO: Connected (version 2.0, client OpenSSH_4.3)
1.168 encoding stdout as sys.stdout encoding 'cp866'
1.168 Traceback (most recent call last):
  File "bzrlib\commands.pyo", line 853, in exception_to_return_code
  File "bzrlib\commands.pyo", line 1055, in run_bzr
  File "bzrlib\commands.pyo", line 661, in run_argv_aliases
  File "bzrlib\commands.pyo", line 665, in run_direct
  File "bzrlib\cleanup.pyo", line 122, in run_simple
  File "bzrlib\cleanup.pyo", line 156, in _do_with_cleanups
  File "bzrlib\builtins.pyo", line 1146, in run
  File "bzrlib\push.pyo", line 83, in _show_push_branch
  File "bzrlib\bzrdir.pyo", line 873, in open_from_transport
  File "bzrlib\transport\__init__.pyo", line 1644, in do_catching_redirections
  File "bzrlib\bzrdir.pyo", line 860, in find_format
  File "bzrlib\bzrdir.pyo", line 1820, in find_format
  File "bzrlib\bzrdir.pyo", line 1830, in probe_transport
  File "bzrlib\transport\sftp.pyo", line 435, in get_bytes
  File "bzrlib\transport\sftp.pyo", line 425, in get
  File "bzrlib\transport\sftp.pyo", line 397, in _get_sftp
  File "bzrlib\transport\sftp.pyo", line 389, in _create_connection
  File "bzrlib\transport\ssh.pyo", line 330, in connect_sftp
  File "bzrlib\transport\ssh.pyo", line 326, in _connect
  File "bzrlib\transport\ssh.pyo", line 540, in _paramiko_auth
  File "bzrlib\config.pyo", line 1288, in get_password
  File "bzrlib\ui\text.pyo", line 132, in get_password
  File "bzrlib\ui\text.pyo", line 208, in prompt
  File "encodings\cp866.pyo", line 12, in encode
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)

[Alexander Belchenko (bialix)]

So, actually, if bzr trying to ask for password and failed to print properly user name on console, I don't think bzr should fail with traceback. encode(user_name, 'replace') should help here, IMO.

[Alexander Belchenko (bialix)]

Actually it affects qbzr too, but it's not clear for me why.

[Alexey Gaynulin (ag-bernoulli)]

1. whoami - i've set it and the bzr whoami command output:
C:\Users\Алексей\Documents\bazar...
Alexey Gaynulin <email address hidden>
2. The password: many times i tested it - doesn't work. Inspite of the fact that password is correct (using this password i login to server - and pass correct). So i don't think that password is wrong...

3. i tried to start again and again:
write command:
C:\Users\Алексей\Documents\bazar-test\experiment\repo\trunk>bzr init-repo --no-trees http://bazaar.09248.ru/bzr/project
bzr: ERROR: Transport operation not possible: http does not support mkdir()

Please advice where is the core of the problem...?

[Alexander Belchenko (bialix)]

http transport is read-only. In your original error you have used sftp:// URL. Please, keep using it.

[Alexander Belchenko (bialix)]

Are you set whoami after I've asked you, or it was set earlier, when you got this error?

[Alexey Gaynulin (ag-bernoulli)]

Alexander i'll try once again from the blank list and inform you...

[Alexey Gaynulin (ag-bernoulli)]

C:\Users\Алексей\Documents\bazar-test\experiment\project>bzr init new/Created a repository tree (format: 2a)
Using shared repository: C:/Users/Алексей/Documents/bazar-test/experiment/project/

C:\Users\Алексей\Documents\bazar-test\experiment\project>cd new

C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr init-repo --no-trees sftp://bazaar.09248.ru/bzr/project
Connected (version 2.0, client OpenSSH_4.3)
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)
............................................

C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr init-repo --no-trees sftp://bazaar.09248.ru/bzr
Connected (version 2.0, client OpenSSH_4.3)
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)
................................................

C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr whoami
Alexey Gaynulin <email address hidden>

C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr checkout sftp://bazaar.09248.ru/bzr
Connected (version 2.0, client OpenSSH_4.3)
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)

As you see, any way i've tried - the same error (insite of correct identification data)....
Don't know what to try next.....

[Alexey Gaynulin (ag-bernoulli)]

By the way - this is the screen shot of everything that is the folder bzr on the server
http://clip2net.com/page/m0/6275643

May be there is lack for something...!?

[Alexey Gaynulin (ag-bernoulli)]

I suppose there is a problem with Russian Cyrilical charset....

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> C:\Users\Алексей\Documents\bazar-test\experiment\project>bzr init new/Created a repository tree (format: 2a)
> Using shared repository: C:/Users/Алексей/Documents/bazar-test/experiment/project/
>
> C:\Users\Алексей\Documents\bazar-test\experiment\project>cd new
>
> C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr init-repo --no-trees sftp://bazaar.09248.ru/bzr/project
> Connected (version 2.0, client OpenSSH_4.3)
> bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 4: ordinal not in range(128)
> ............................................

Please, do following:

set BZR_PDB=1
bzr init-repo --no-trees sftp://bazaar.09248.ru/bzr/project

This time when error occurred you will enter debugger mode.
In the debugger mode execute following command:

 >>> print repr(input)

And show me its result.

[Alexey Gaynulin (ag-bernoulli)]

DONE:

...
 .... including this traceback and a description of the problem.
**** entering debugger
> c:\users\рыхъёхщ\documents\bazar-test\experiment\project\new\encodings\cp866.p
yo(12)encode()
(Pdb) >>> print repr(input)
*** SyntaxError: invalid syntax (<stdin>, line 1)
(Pdb)

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> DONE:
>
>
> ...
> .... including this traceback and a description of the problem.
> **** entering debugger
>> c:\users\рыхъёхщ\documents\bazar-test\experiment\project\new\encodings\cp866.p
> yo(12)encode()
> (Pdb) >>> print repr(input)
> *** SyntaxError: invalid syntax (<stdin>, line 1)
> (Pdb)
>

One more time, but this time without leading >>> (sorry for confusion).
Just type

print repr(input)

and hit Enter. You should see some string.

[Alexey Gaynulin (ag-bernoulli)]

Alexander, please:

    including this traceback and a description of the problem.
**** entering debugger
> c:\encodings\cp866.pyo(12)encode()
(Pdb)
(Pdb) print repr(input)
'SSH \xc0\xeb\xe5\xea\xf1\xe5\<email address hidden> password: '
(Pdb)

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> Alexander, please:
>
> including this traceback and a description of the problem.
> **** entering debugger
>> c:\encodings\cp866.pyo(12)encode()
> (Pdb)
> (Pdb) print repr(input)
> 'SSH \xc0\xeb\xe5\xea\xf1\xe5\<email address hidden> password: '
> (Pdb)
>

Ha! finally we get to the root of the problem!

OK, so \xc0\xeb\xe5\xea\xf1\xe5\<email address hidden> string is actually

Алексей@bazaar.09248.ru in CP1251.

So, my best guess that bzr uses your windows login name (Алексей) as the
username to login to your sftp server. And because of non-ascii
characters you got this traceback.

We need to fix this in bzr.

But for you there is very simple workaround: specify your login name on
remote aftp server with following syntax:

sftp://<email address hidden>/...

and it will work.

Also you can set this name in authentication.conf as following:

[bazaar.09248.ru]
host = bazaar.09248.ru
name = xxx
scheme = ssh

and then bzr will use this predefined name even if you don't specify it
in the sftp:// URL.

Thank you for bug report and your help in debugging.
I will need more of your help tomorrow to understand why QBzr failing too.

[Alexey Gaynulin (ag-bernoulli)]

So the result is:

"C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr init-repo --no-
trees sftp://<email address hidden>/bzr/
Connected (version 2.0, client OpenSSH_4.3)
SSH <email address hidden> password:
Authentication (password) successful!
Secsh channel 1 opened.
bzr: ERROR: Unable to start sftp client bazaar.09248.ru; EOF during negotiation"

This time looks much better. I'ld say "looks nice" but have some mistake (sure - this one is caused by some of not corrected actions).
  "... EOF during negotiation"
Tommorrow will try once again from begining - hope there won't any problem.

i'll do my best to help you with QBzr failing.

Thank you, Alexander!
Хорошего вечера! )

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> So the result is:
>
>
> "C:\Users\Алексей\Documents\bazar-test\experiment\project\new>bzr init-repo --no-
> trees sftp://<email address hidden>/bzr/
> Connected (version 2.0, client OpenSSH_4.3)
> SSH <email address hidden> password:
> Authentication (password) successful!
> Secsh channel 1 opened.
> bzr: ERROR: Unable to start sftp client bazaar.09248.ru; EOF during negotiation"

Strange error. Check if your sftp/ssh server is actually running. And
show the relevant part of .bzr.log, as usual ;-)

>
>
> This time looks much better. I'ld say "looks nice" but have some mistake (sure - this one is caused by some of not corrected actions).
> "... EOF during negotiation"
> Tommorrow will try once again from begining - hope there won't any problem.
>
> i'll do my best to help you with QBzr failing.
>
> Thank you, Alexander!
> Хорошего вечера! )

Спасибо, тебе тоже.

[Alexey Gaynulin (ag-bernoulli)]

Alexander, good day!

Concerning the following error:

C:\Users\Алексей\Documents\bazar-test\experiment\test>bzr init-repo --no-trees s
ftp://<email address hidden>/bzr/PROJECT
Connected (version 2.0, client OpenSSH_4.3)
SSH <email address hidden> password:
Authentication (password) successful!
Secsh channel 1 opened.
bzr: ERROR: Unable to start sftp client bazaar.09248.ru; EOF during negotiation

Send you the logs (attached)

[Alexey Gaynulin (ag-bernoulli)]

Please, advice whether it is server problem or bazaar?

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> Please, advice whether it is server problem or bazaar?

It seems like the problem on the server. Can you check with any
standalone ssh/sftp client that you actually can connect to your server
over sftp? You can use WinSCP or Putty, or something else.

[Alexey Gaynulin (ag-bernoulli)]

ssh/sftp are valid on the server...

[Alexey Gaynulin (ag-bernoulli)]

try through the putty - the same problem. Ask Server admin to check rights of the user - the required connection was denied.
Thanks for help! Now it works.

i'm going to continue to learn it up...

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> ssh/sftp are valid on the server...
>

Then... I don't know, sorry.

This error
bzr: ERROR: Unable to start sftp client bazaar.09248.ru; EOF during
negotiation
is different one from original UnicodeError and therefore I suggest you
to ask how to resolve it via Answers section or file a new bug.

Thanks.

[Alexey Gaynulin (ag-bernoulli)]

Alexander, as i wrote:
"Ask Server admin to check rights of the user - the required connection was denied."

Now it works. Everything is ok! Thanks!

[Alexander Belchenko (bialix)]

Alexey Gaynulin пишет:
> try through the putty - the same problem. Ask Server admin to check rights of the user - the required connection was denied.
> Thanks for help! Now it works.
>
> i'm going to continue to learn it up...

In the case you don't know about Russian resources:
http://groups.google.com/group/ru_bzr
http://bzr-day.blogspot.com/

HTH

[Stas Kalashnikov (halega)]

I have the same problem: I can't get the branch if the user's folder name is in Russian (Windows 7 Home Edititon, Russian):

bzr branch ftp://server/shared-repo/branch1

bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4
in position 4: ordinal not in range(128)

Traceback (most recent call last):
 File "bzrlib\commands.pyo", line 911, in exception_to_return_code
 File "bzrlib\commands.pyo", line 1111, in run_bzr
 File "bzrlib\commands.pyo", line 689, in run_argv_aliases
 File "bzrlib\commands.pyo", line 704, in run
 File "bzrlib\cleanup.pyo", line 135, in run_simple
 File "bzrlib\cleanup.pyo", line 165, in _do_with_cleanups
 File "bzrlib\builtins.pyo", line 1209, in run
 File "bzrlib\bzrdir.pyo", line 1032, in open_tree_or_branch
 File "bzrlib\bzrdir.pyo", line 911, in open
 File "bzrlib\bzrdir.pyo", line 941, in open_from_transport
 File "bzrlib\lazy_import.pyo", line 125, in __call__
 File "bzrlib\transport\__init__.pyo", line 1657, in do_catching_redirections
 File "bzrlib\bzrdir.pyo", line 928, in find_format
 File "bzrlib\bzrdir.pyo", line 1949, in find_format
 File "bzrlib\bzrdir.pyo", line 1959, in probe_transport
 File "bzrlib\transport\__init__.pyo", line 610, in get_bytes
 File "bzrlib\transport\ftp\__init__.pyo", line 248, in get
 File "bzrlib\transport\ftp\__init__.pyo", line 108, in _get_FTP
 File "bzrlib\transport\ftp\__init__.pyo", line 141, in _create_connection
 File "bzrlib\transport\ftp\__init__.pyo", line 158, in _login
 File "bzrlib\config.pyo", line 1236, in get_password
 File "bzrlib\ui\text.pyo", line 128, in get_password
 File "bzrlib\ui\text.pyo", line 204, in prompt
 File "encodings\cp866.pyo", line 12, in encode
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 4: ordinal
not in range(128)

bzr 2.2.0 on python 2.6.4 (Windows-post2008Server-6.1.7600)
arguments: ['bzr', 'branch', 'ftp://silver/vbk-repo/trunk']
encoding: 'cp1251', fsenc: 'mbcs', lang: None
plugins:
 bzrtools C:\Program Files (x86)\Bazaar\plugins\bzrtools [2.2.0]
 colo C:\Program Files (x86)\Bazaar\plugins\colo [0.1.0]
 explorer C:\Program Files (x86)\Bazaar\plugins\explorer [1.1b1]
 fastimport C:\Program Files (x86)\Bazaar\plugins\fastimport [0.9.0de
v]
 launchpad C:\Program Files (x86)\Bazaar\plugins\launchpad [2.2.0]
 loom C:\Program Files (x86)\Bazaar\plugins\loom [2.2.1dev]
 netrc_credential_store C:\Program Files (x86)\Bazaar\plugins\netrc_credential_
store [2.2.0]
 news_merge C:\Program Files (x86)\Bazaar\plugins\news_merge [2.2.0]
 pipeline C:\Program Files (x86)\Bazaar\plugins\pipeline [unknown]
 qbzr C:\Program Files (x86)\Bazaar\plugins\qbzr [0.19.0]
 rewrite C:\Program Files (x86)\Bazaar\plugins\rewrite [0.6.0]
 svn C:\Program Files (x86)\Bazaar\plugins\svn [1.0.3]
 upload C:\Program Files (x86)\Bazaar\plugins\upload [1.0.0dev]
 xmloutput C:\Program Files (x86)\Bazaar\plugins\xmloutput [0.8.6]

*** Bazaar has encountered an internal error. This probably indicates a
   bug in Bazaar. You can help us fix it by filing a bug rep...