Qt frontend for Bazaar

crash with winmerge in cp932 japanese character

Reported by Heart-Rhythm on 2010-02-18
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Bazaar
Medium
INADA Naoki
QBzr
High
INADA Naoki
TortoiseBZR
Undecided
Unassigned

Bug Description

bzr: ERROR: exceptions.UnicodeDecodeError: 'cp932' codec can't decode bytes in position 26-27: illegal multibyte sequence

Traceback (most recent call last):
  File "C:/Program Files/Bazaar/plugins\qbzr\lib\subprocess.py", line 605, in readStdout
UnicodeDecodeError: 'cp932' codec can't decode bytes in position 26-27: illegal multibyte sequence

bzr 2.1.0 on python 2.5.4 (win32)
arguments: ['C:\\Program Files\\Bazaar\\bzr.exe', '--command=diff', '--file=C:\\TEMP\\bzrtest-bugfix\\\x83_\x83~\x81[\x95\\\x83h\x83L\x83\x85\x83\x81\x83\x93\x83g.txt']
encoding: 'cp932', fsenc: 'mbcs', lang: None
plugins:
  bzrtools C:\Program Files\Bazaar\plugins\bzrtools [2.1.0]
  explorer C:\Program Files\Bazaar\plugins\explorer [1.0.0rc1]
  launchpad C:\Program Files\Bazaar\plugins\launchpad [2.1.0]
  netrc_credential_store C:\Program Files\Bazaar\plugins\netrc_credential_store [2.1.0]
  qbzr C:\Program Files\Bazaar\plugins\qbzr [0.18.1]
  rebase C:\Program Files\Bazaar\plugins\rebase [0.5.5]
  svn C:\Program Files\Bazaar\plugins\svn [1.0.2]
  upload C:\Program Files\Bazaar\plugins\upload [1.0.0dev]
  xmloutput C:\Program Files\Bazaar\plugins\xmloutput [0.8.6]

Related branches

lp:~songofacandy/bzr/fix-523746-2
Rejected for merging into lp:bzr/2.0
Vincent Ladeuil: Needs Information on 2010-03-03
Martin Packman: Needs Information on 2010-02-25
lp:~songofacandy/bzr/fix-523746-dev
Merged into lp:bzr at revision 5892
John A Meinel: Approve on 2011-05-16
Vincent Ladeuil: Abstain on 2010-04-22
Alexander Belchenko: Pending requested 2011-05-06
Gary van der Merwe: Pending requested 2011-05-06
Martin Packman: Pending requested 2011-05-06
bzr-core: Pending requested 2010-04-29
Alexander Belchenko (bialix) wrote :

Naoki, I need your help on understanding what's going on.

Heart-Rhythm さん、バグ報告ありがとうございます。
問題解決のために、問題発生時の詳しい状況を教えていただけますか?

1. WinMerge は Unicode版 (WinMergeU) でしょうか? ASCII 版でしょうか?

2. どういった操作をしたときにこのエラーが発生したのでしょうか?

3. スタートメニューから "Open bzr log file" を選ぶと、詳細なログが表示されるので、
   該当するエラーの周辺を添付していただけますか?

INADA Naoki (songofacandy) wrote :

すみません、試してみたら簡単に再現できました。

2010/2/18 INADA Naoki <email address hidden>:
> Heart-Rhythm さん、バグ報告ありがとうございます。
> 問題解決のために、問題発生時の詳しい状況を教えていただけますか?
>
> 1. WinMerge は Unicode版 (WinMergeU) でしょうか? ASCII 版でしょうか?
>
> 2. どういった操作をしたときにこのエラーが発生したのでしょうか?
>
> 3. スタートメニューから "Open bzr log file" を選ぶと、詳細なログが表示されるので、
> 該当するエラーの周辺を添付していただけますか?
>

--
INADA Naoki <email address hidden>

INADA Naoki (songofacandy) wrote :
Download full text (4.5 KiB)

I success to reproduce this bug:

C:\test2>bzr init
Created a standalone tree (format: 2a)

C:\test2>echo abc > あいう

C:\test2>bzr add あいう
adding "あいう"

C:\test2>bzr commit -m "abc"
Committing to: C:/test2/
added あいう
Committed revision 1.

C:\test2>echo def >> あいう

C:\test2>bzr diff
=== modified file '縺ゅ>縺・
--- 縺ゅ>縺・2010-02-18 12:12:23 +0000
+++ 縺ゅ>縺・2010-02-18 12:12:40 +0000
@@ -1,1 +1,2 @@
 abc
+def

C:\test2>bzr diff --using=C:\\usr\\WinMerge\\WinMergeU.exe
=== modified file '縺ゅ>縺・
bzr: ERROR: exceptions.UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-86: ordinal not in r
(128)

Traceback (most recent call last):
  File "bzrlib\commands.pyo", line 853, in exception_to_return_code
  File "bzrlib\commands.pyo", line 1055, in run_bzr
  File "bzrlib\commands.pyo", line 661, in run_argv_aliases
  File "bzrlib\commands.pyo", line 665, in run_direct
  File "bzrlib\cleanup.pyo", line 122, in run_simple
  File "bzrlib\cleanup.pyo", line 156, in _do_with_cleanups
  File "bzrlib\commands.pyo", line 1070, in ignore_pipe
  File "bzrlib\builtins.pyo", line 1955, in run
  File "bzrlib\diff.pyo", line 444, in show_diff_trees
  File "bzrlib\diff.pyo", line 889, in show_diff
  File "bzrlib\diff.pyo", line 949, in _show_diff
  File "bzrlib\diff.pyo", line 975, in _diff
  File "bzrlib\diff.pyo", line 506, in _diff_many
  File "bzrlib\diff.pyo", line 782, in diff
  File "bzrlib\diff.pyo", line 708, in _execute
  File "subprocess.pyo", line 594, in __init__
  File "subprocess.pyo", line 822, in _execute_child
UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-86: ordinal not in range(128)

bzr 2.1.0 on python 2.5.4 (Windows-XP-5.1.2600-SP3)
arguments: ['bzr', 'diff', '--using=C:\\\\usr\\\\WinMerge\\\\WinMergeU.exe']
encoding: 'cp932', fsenc: 'mbcs', lang: 'ja'
plugins:
  bzrtools C:\Program Files\Bazaar\plugins\bzrtools [2.1.0]
  explorer C:\Documents and Settings\inada-n\Application Data\bazaar\2.0\plugins\explorer [1.0.0.rc.0]
  launchpad C:\Program Files\Bazaar\plugins\launchpad [2.1.0]
  loom C:\Documents and Settings\inada-n\Application Data\bazaar\2.0\plugins\loom [1.4.0dev]
  netrc_credential_store C:\Program Files\Bazaar\plugins\netrc_credential_store [2.1.0]
  qbzr C:\Documents and Settings\inada-n\Application Data\bazaar\2.0\plugins\qbzr [0.19.0dev]
  rebase C:\Program Files\Bazaar\plugins\rebase [0.5.5]
  scmproj C:\Documents and Settings\inada-n\Application Data\bazaar\2.0\plugins\scmproj [0.6.0dev]
  svn C:\Program Files\Bazaar\plugins\svn [1.0.2]
  symlink C:\Documents and Settings\inada-n\Application Data\bazaar\2.0\plugins\symlink [0.92.0]
  upload C:\Program Files\Bazaar\plugins\upload [1.0.0dev]
  xmloutput C:\Program Files\Bazaar\plugins\xmloutput [0.8.6]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

Traceback is:
0.093 bazaar version: 2.1.0
0....

Read more...

INADA Naoki (songofacandy) wrote :

I can reproduce this bug without qbzr.

C:\test2>bzr --no-plugins diff --using=C:\\usr\\WinMerge\\WinMergeU.exe
=== modified file '縺ゅ>縺・
bzr: ERROR: exceptions.UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-86: ordinal not in range
(128)

Traceback (most recent call last):
  File "bzrlib\commands.pyo", line 853, in exception_to_return_code
  File "bzrlib\commands.pyo", line 1055, in run_bzr
  File "bzrlib\commands.pyo", line 661, in run_argv_aliases
  File "bzrlib\commands.pyo", line 665, in run_direct
  File "bzrlib\cleanup.pyo", line 122, in run_simple
  File "bzrlib\cleanup.pyo", line 156, in _do_with_cleanups
  File "bzrlib\commands.pyo", line 1070, in ignore_pipe
  File "bzrlib\builtins.pyo", line 1955, in run
  File "bzrlib\diff.pyo", line 444, in show_diff_trees
  File "bzrlib\diff.pyo", line 889, in show_diff
  File "bzrlib\diff.pyo", line 949, in _show_diff
  File "bzrlib\diff.pyo", line 975, in _diff
  File "bzrlib\diff.pyo", line 506, in _diff_many
  File "bzrlib\diff.pyo", line 782, in diff
  File "bzrlib\diff.pyo", line 708, in _execute
  File "subprocess.pyo", line 594, in __init__
  File "subprocess.pyo", line 822, in _execute_child
UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-86: ordinal not in range(128)

bzr 2.1.0 on python 2.5.4 (Windows-XP-5.1.2600-SP3)
arguments: ['bzr', '--no-plugins', 'diff', '--using=C:\\\\usr\\\\WinMerge\\\\WinMergeU.exe']
encoding: 'cp932', fsenc: 'mbcs', lang: 'ja'
plugins:

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

INADA Naoki (songofacandy) wrote :

OK, There are three problems.

1. bzr diff shows filename in "utf-8" encoding.
    This is not bad if stdout redirected to file. But if stdout is console, bzr should encode with stdout.encoding.

2. bzr diff --using=prog fails when arguments contains non ascii char.

3. qbzr crashes when qsubprocess writes binary to stdout.
    I think qbzr should use decode with errors="replace" or errors="backslashreplace".

Changed in tortoisebzr:
status: New → Invalid

Heart-Rhythm пишет:
> Public bug reported:
>
> bzr: ERROR: exceptions.UnicodeDecodeError: 'cp932' codec can't decode
> bytes in position 26-27: illegal multibyte sequence
>
> Traceback (most recent call last):
> File "C:/Program Files/Bazaar/plugins\qbzr\lib\subprocess.py", line 605, in readStdout
> UnicodeDecodeError: 'cp932' codec can't decode bytes in position 26-27: illegal multibyte sequence
>
> bzr 2.1.0 on python 2.5.4 (win32)
> arguments: ['C:\\Program Files\\Bazaar\\bzr.exe', '--command=diff', '--file=C:\\TEMP\\bzrtest-bugfix\\\x83_\x83~\x81[\x95\\\x83h\x83L\x83\x85\x83\x81\x83\x93\x83g.txt']
> encoding: 'cp932', fsenc: 'mbcs', lang: None

Please, show me relevant part of .bzr.log. There should be more info
about this problem.

Alexander Belchenko (bialix) wrote :

Thanks, Naoki.

INADA Naoki (songofacandy) wrote :

This is a patch to fix problem 2.

BTW, Should we split this bug to 3 bugs?

INADA Naoki (songofacandy) wrote :

This patch fixes problem 3.

INADA Naoki (songofacandy) wrote :

I can't fix problem 1 because I don't know about ui well.
How am I able to determine outf is redirected?

INADA Naoki (songofacandy) wrote :

I've committed fixing problem 2 to lp:qbzr and lp:qbzr/0.18.

Changed in qbzr:
milestone: none → 0.18.2
assignee: nobody → INADA Naoki (songofacandy)
status: New → Fix Released
Changed in bzr:
status: New → Confirmed
Alexander Belchenko (bialix) wrote :

Naoki, fixing problem 1 should be done in bzr itself. There is bug report about utf-8.

But unfortunately we have no way to detect if we redirect to file or simply run `bzr command|less`.

Thanks for your patches.

Changed in qbzr:
importance: Undecided → High
INADA Naoki (songofacandy) wrote :

Should I make a branch from bzr-2.0?

INADA Naoki (songofacandy) wrote :

I've reimplemented fix and propose merge request.

Martin Pool (mbp) on 2010-03-22
Changed in bzr:
assignee: nobody → INADA Naoki (songofacandy)
status: Confirmed → In Progress
John A Meinel (jameinel) on 2011-04-21
tags: added: patch-needswork unicode
Changed in bzr:
importance: Undecided → Medium
Martin Packman (gz) wrote :

The branch addressing this problem has landed! Some additional manual testing may be useful here to ensure there aren't any regressions.

Changed in bzr:
milestone: none → 2.4b3
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers