Unicode display and input is broken.

[Michael B. Trausch]

When using GNOME Terminal in Hardy, the Unicode display and input is broken. Some characters (e.g., «, », ©, ®, ™, and others) are transliterated, and others still (é, á, í, ö, and more accented letters) simply show up as "?". ß shows up as ss, and so forth. This is horribly broken, /especially/ when I am entering a unicode code point and expect a UTF-8 output and instead get an ASCII transliteration. These may very well be valid transliterations, but they are *not* desired; if I wanted to use transliterations of Unicode characters, I would type those in instead!

Along the same lines, man pages no longer display correctly unless I invoke man with a modified environment:

  LC_ALL=C man ls

works fine, but

  man ls

The latter does not show apostrophes, some hyphens, quotation marks, and other UTF-8 encoded characters.

[Michael B. Trausch (mtrausch)]

Side note: "LC_ALL=POSIX man ls" works as well.

[Michael B. Trausch (mtrausch)]

Better and quicker test case:

mbt@garlic:~$ apt-cache showsrc vte
Package: vte
Binary: python-vte, libvte-doc, python-vte-dbg, libvte-dev, libvte-common, libvte9
Version: 1:0.16.11-1ubuntu1
Priority: optional
Section: libs
Maintainer: Ubuntu Core developers <email address hidden>
Original-Maintainer: Guilherme de S. Pastore <email address hidden>
Build-Depends: cdbs (>= 0.4.43), debhelper (>= 5.0.37.2), libglib2.0-dev (> 2.9.0), libgtk2.0-dev (>= 2.6.0), libgtk-directfb-2.0-dev (>= 2.6.0), gettext, python-gtk2-dev, python-all-dev (>= 2.3.5-11), python-all-dbg, libncurses5-dev, libxrender-dev, gnome-pkg-tools (>= 0.10), dpkg-dev (>= 1.13.19), gtk-doc-tools, libxt-dev, libfreetype6-dev (>= 2.0.2), python-support (>= 0.3), libpango1.0-dev, libxft-dev
Architecture: any
Standards-Version: 3.7.2
Format: 1.0
Directory: pool/main/v/vte
Files:
 bcf3ed65fd212bfc6fda015e159f10b8 1344 vte_0.16.11-1ubuntu1.dsc
 daed00f149111ef4d022f3d7d0f03c61 1398847 vte_0.16.11.orig.tar.gz
 faa5a790859ba137c34582d67e6122ac 17689 vte_0.16.11-1ubuntu1.diff.gz
Uploaders: Debian GNOME Maintainers <email address hidden>, Loic Minier <email address hidden>, Sebastian Dr��ge <email address hidden>
Python-Version: >= 2.4
Vcs-Bzr: http://code.launchpad.net/~ubuntu-desktop/vte/ubuntu

mbt@garlic:~$

[Sebastien Bacher (seb128)]

Does it work when using an another command line software?

[Michael B. Trausch (mtrausch)]

On Tue, 2008-01-08 at 09:10 +0000, Sebastien Bacher wrote:
> Does it work when using an another command line software?

Of course. Nothing changed in xterm. It still works just fine. Of
course, it does not support artibrary unicode entry (C-S-u ####), but it
supports everything else.

 --- Mike

--
Michael B. Trausch <email address hidden>
home: 404-592-5746, 1 www.trausch.us
cell: 678-522-7934 im: <email address hidden>, jabber
Ubuntu Unofficial Backports Project: http://backports.trausch.us/

[Michael B. Trausch (mtrausch)]

Also, please note that this is new behavior in Hardy. In Feisty and Gutsy, gnome-terminal handled UTF-8/Unicode input and output correctly—e.g., not displaying diamond-question marks for UTF-8 characters, and not transliterating, ignoring, or mishandling Unicode input by Unicode code point in the way that GNOME permits arbitrary Unicode input.

In Gutsy, for example, you could press C-S-u 2122 and you would get the trademark symbol (“™”), or C-S-u 3c0 and get pi (“π”). Presently, you get a transliterated trademark symbol (“(TM)”) and a question mark (“?”), respectively. This is a definite step backwards in an international world…

[Vytas (vytas)]

Same here. I can't neither enter nor paste Lithuanian latin symbols like
ĄČĘĖĮŠŲŪ

You can try pasting these and reproduce the bug - only ? (question marks) are displayed

[Hans Deragon (deragon)]

To fix this problem, you can set in /etc/environment :

LANG="en_CA.UTF-8"

Of course, add your own language/country code. Reboot so the change can take effect.

This bug should remain open however; there should be a default "LANG=<language/country selected by user>.UTF-8". UTF-8 should be the default, out of the box.

[Vytas (vytas)]

This seems to be fixed (at least for me) with latest hardy updates, can anyone confirm?

[Michael B. Trausch (mtrausch)]

Fix is confirmed.