Cursor in terminal behaves badly with special characters present

[Don Cristóbal]

Binary package hint: xfce4-terminal

Both xfce4-terminal and gnome-terminal are affected, but not the text-only terminals (alt+f1, f2 etc.). So probably the package hint is wrong...

Description:
With special characters present before the $ sign, when going back to the previous line (backspace or cursor keys), the cursor jumps to the left too far (one character per special character).

How to reproduce (cf. always my attachment):
1. Open the terminal (in X, not the real text-only screens)
2. Create a folder with special characters in it, e.g. with german special characters: $ mkdir abcäöü
3. Go to the special character folder. $ cd abcäöü
4. Type some rubbish until you reach the end of the screen (in my attached example: "abcdefghijklmnopqrst". As soon as you have filled up the last column (the "t" in my case), the cursor jumps correctly to the next line.
5. Hit backspace or the left cursor once. Backspace eats 2 characters instead of one if one special character is present (3 if there are 2; 4 if there are 3 etc.).
6. Hit return. Bash says it does not know the rubbish that you've actually typed (not the rubbish that was displayed). So obviously this is a display problem, not an input problem.

Instead of step 4, you can also type some new rubbish. It will be inserted into the old rubbish exactly the way you type it, but it won't display correctly. It's like a blind flight: As long as you know exactly where you are, you'll land on the airport. But never try to tell your position from what you see...

The bug is most probably not a duplicate of Bug #238055 or Bug #88504. The symptoms described there don't match ours here.

Xubuntu 9.04, up to date as of July 15 2009.

$ apt-cache policy gnome-terminal
gnome-terminal:
  Installed: 2.26.0-0ubuntu2
  Candidate: 2.26.0-0ubuntu2
  Version table:
 *** 2.26.0-0ubuntu2 0
        500 http://ch.archive.ubuntu.com jaunty/main Packages
        100 /var/lib/dpkg/status

$ apt-cache policy xfce4-terminal
xfce4-terminal:
  Installed: 0.2.10-1ubuntu2
  Candidate: 0.2.10-1ubuntu2
  Version table:
 *** 0.2.10-1ubuntu2 0
        500 http://ch.archive.ubuntu.com jaunty/universe Packages
        100 /var/lib/dpkg/status

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
Package: xfce4-terminal 0.2.10-1ubuntu2
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: xfce4-terminal
Uname: Linux 2.6.28-11-generic i686

WORKAROUND:
1. The correct text can be displayed by hitting the up and then the down cursor button.
2. The bug is triggered by special characters in what comes before the $ sign. If those special
     characters are removed from the prompt, the terminal acts correctly.
     Please see comments #9 and #10.

[Don Cristóbal (doncristobal)]

My screenshot. I underlined the "s" and the place where it should be displayed, but is not.

[Charlie Kravetz (cjkgeek)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately we can't fix it without more information. I have attempted to reproduce this in Xubuntu 9.04. I could not reproduce it in Xfce4-terminal, so I added gnome-terminal. I still can not reproduce it. Do you have any compositors in use? If you are using compiz, can you turn it off and see if this bug exists. Also, please attach your ~/.xsession-errors log file as a plain text file. Thanks again.

[Don Cristóbal (doncristobal)]

No compiz, no pseudo-fancy effects, just a nice and clean Xfce. Maybe Scim is the culprit (once again)?

Please request any other information you need, I'll be happy to (try and) provide it. Thanks for taking care of the subject! You can imagine that it's rather annoying.

[Don Cristóbal (doncristobal)]

Who-hooo, I've just discovered that compiz is actually installed!
compiz:
  Installed: (none)
  Candidate: 1:0.8.2-0ubuntu8
  Version table:
     1:0.8.2-0ubuntu8 0
        500 http://ch.archive.ubuntu.com jaunty/main Packages

I always thought that was only for silly rotating screen effects and the like... eating your memory and cpu for nothing... never expected it to be in a Xubuntu.
How do I find out if it's active in some way or another? I only find Settings > Window Manager Tweaks > Compositor: Everything disabled

[Charlie Kravetz (cjkgeek)]

You were right the first time.
  Installed: (none)
  Candidate: 1:0.8.2-0ubuntu8
So if you installed it, it would install the Candidate: version. Installed shows none.

Thanks for the answers. I am marking this triaged to give the developers a chance to see if we can fix it.

[Don Cristóbal (doncristobal)]

Just one additional piece of information: The correct text can be displayed by hitting the up and then the down cursor button. This confirms that the terminal gets the right input and then just displays it the wrong way.

And, at least, it's a way to survive out there in the jungle, when you've lost your path...

[Don Cristóbal (doncristobal)]

If someone can tell me how to temporarily deactivate scim (without uninstalling it), I'll also check if the error occurs without it.
* Right-click - Exit just restarts scim
* In the scim setup dialog, I don't see any such option
* I don't find scim in the services list from Applications - System - Services, or in the Session and Startup dialog from Applications - Settings
* Searching the web somehow did not help either.

[Don Cristóbal (doncristobal)]

I found a new trace! The bug is triggered by special characters in what comes before the $ sign.

E.g.:
~/Musik/Nik Baertsch's Ronin/Holon$
Error does not happen.

~/Musik/Nik Bärtschs Ronin/Holon$
Error happens. This also shows that only the german Umlaut ä causes it, and not the single quote/apostrophe. Can anyone try to confirm this on other machines?

[Don Cristóbal (doncristobal)]

One more detail: Each special character before the $ sign eats one character. This seems to be a unicode problem with one and two byte characters.

E.g.
/media/data/Musik/aaaaaäaaa/ooooööooo$ # eats 3 characters when going backwards
/media/data/Musik/äüöäüöääöäüöäü$ # eats /many/ characters
/media/data/Musik/aaaaaäaaa$ # eats 1 character

(Sorry for the mail bombing)

[Rolf Leggewie (r0lf)]

Thanks to the excellent analysis done by Christoph in comment 10, I can confirm this is still present even in the bleeding edge alpha packages from 0.2.90-1 from Debian Experimental recompiled for Karmic.

[Rolf Leggewie (r0lf)]

I can also confirm this for the current gnome-terminal from Karmic. This makes me suspect it is not a bug in either of the two but in a commonly used library (vte?).

My first hunch was that this may be a problem in vte, even more so after going through the dependencies of gnome-terminal and xfce4-terminal. But after reading through the vte bug tracker and confirming that this problem also exists in xterm which does not use libvte, I came to the conclusion it must be somewhere else.

I'll add gnome-terminal and xterm as being affected now although I don't think the bug itself is in either of those packages.

[Rolf Leggewie (r0lf)]

FWIW, I can confirm that this is not dependent on scim running. I know scim is quite buggy, but this time the culprit is elsewhere ;-)

[Thomas Dickey (dickey-his)]

From the description, it sounds as if the problem is in bash.

[Bryce Harrington (bryce)]

Moving to bash as per comment #14

[Charlie Kravetz (cjkgeek)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. The issue you are reporting is an upstream one and it would be nice if somebody having it could send the bug to the developers of the software by using 'bashbug' in a terminal. If you have done so, please tell us the number of the upstream bug (or the link), so we can add a bugwatch that will inform us about its status. Thanks in advance.

[Don Cristóbal (doncristobal)]

I (original reporter) have reported the bug upstream, using bashbug... but I fear was silly not to include my email address so they wont be able to send me any link or anything... However, I included the Launchpad bug number, so maybe someone finds the other way round :-)
Charlie, are you sure you don't have it? As far as I know it happens on every system, as soon as you use special characters. If you need some to copy&paste, there you go: äöüëïóñÑÒÂ. Tell me if you'd like some more :-)

[Thaddaeus Tintenfisch (thad-fisch-deactivatedaccount)]

There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. It would help us a lot if you could test it on a currently supported Ubuntu version.

[Launchpad Janitor (janitor)]

[Expired for xfce4-terminal (Ubuntu) because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for bash (Ubuntu) because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for gnome-terminal (Ubuntu) because there has been no activity for 60 days.]