When characters not in Ubuntu Mono appear on a line, Konsole cuts off half a letter

[dotancohen]

I have a non-ASCII character leading my Bash prompt. When using the otherwise terrific UbuntuMono font half of the last letter in my CLI is cut off. See attached screenshot. The top line is from root, which has the stock Bash prompt. The lower line is from my user, with the non-ASCII character leading the prompt.

✈saturn:~$ cat /etc/issue
Ubuntu 11.10 \n \l

✈saturn:~$ uname -a
Linux saturn 3.0.0-16-generic #28-Ubuntu SMP Fri Jan 27 17:44:39 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

✈saturn:~$ kde4-config --version
Qt: 4.7.4
KDE Development Platform: 4.7.4 (4.7.4)
kde4-config: 1.0

[dotancohen (dotancohen)]

Attached is a screenshot of the two debug symbols under four conditions:
The first line is on the root user with the stock Bash prompt. First we see the two characters with a space between them, then again one after the other.
The second line is on the regular user with a non-ASCII character leading the Bash prompt. First we see the two characters with a space between them, then again one after the other.

[dotancohen (dotancohen)]

The the special debugging characters are U+EFFD and U+F000 (‘’ and ‘’) as mentioned on this post:
http://design.canonical.com/2011/09/ubuntu-monospace-beta/

[Adolfo Jayme Barrientos (fitojb)]

I think this is not a problem with the Ubuntu Mono fonts, but with Gnome Terminal (or whatever other terminal that you're using). This is caused because your unicode special character is wider than the other characters (because the plane is not included in Ubuntu Mono it is being substituted from a wider font) and causes spacing issues in the application. Please test this using another monospaced font and I'm pretty sure you will see the same issue.

[Kde-2011-08 (kde-2011-08)]

Created attachment 70114
Screenshot depicting issue.

When a character wider than the monospace font is used in Konsole, the cursor appears to cover half of the last letter of the line. See attached screenshot. The top line of the screenshot is from root, which has the stock Bash prompt. The lower line is from my user, which has a wide non-ASCII character leading the prompt.

Note that this issue is not limited to only non-ASCII dingbats like that used in the screenshot. My own native language is not represented in ASCII either, and may be displayed using a font that is wider than the ASCII font used for ASCII.

[dotancohen (dotancohen)]

Thank you, I filed upstream here:
https://bugs.kde.org/show_bug.cgi?id=297390

[adaptee (adaptee)]

Could you provide the Unicode code point for that leading symbol in the screenshot? like U+4E2D for "中", U+305F for "た" ?

Your screenshot might be another example of the problem tracked in bug 41744(a very old bug, I know)

[Kde-2011-08 (kde-2011-08)]

Thank you, it is U+2708
http://www.fileformat.info/info/unicode/char/2708/index.htm

It looks plausible that this is a case of bug 41744, but I am not knowledgeable enough to say for certain either way. However, certainly some of the last comments on that bug are conflated with this issue.

[adaptee (adaptee)]

Thanks for the infomation.

It truns out my previous guessing may be wrong. That symbol is defined as narrow(actually neutral)according to Unicode data[1][2]. However the screenshot shows that it is displayed as Wide, taking the width of two cells. I can reproduce the problem when using some fonts, while not when using other fonts. At the moment I'm not sure whether this is a problem of fonts or a problem of Konsole.

which font is used in your screenshot?

[1] http://www.unicode.org/Public/UNIDATA/EastAsianWidth.txt
[2] http://unicode.org/reports/tr11/

[Kde-2011-08 (kde-2011-08)]

The font in the screenshot was Ubuntu Mono. I won't be at my computer for a few weeks to check, but I can confirm that the phenomenon is in fact font-dependant. All the fonts that I like suffer this problem, but some very ugly fonts do not.

Here is the original bug that I filed on the Ubuntu Mono font:
https://bugs.launchpad.net/ubuntu-font-family/+bug/958142

[Rannasingha Aarachchigee Sisikossal Chhathranga Rannasingha (rascranasingha)]

I suffer from the same problem with certain unicode characters and can confirm this. However I'd like to add that just like Cohen said, it's dependent on the font that's used. I've recently tried to set Monaco (which is wider than the standard font used) as my Konsole font, but found that konsole eats up a little of the prompt when I set the font to that. I'm thinking it's got something to do with the character width.

Here's a screenshot:

http://imageshack.us/photo/my-images/823/xamc.png/

[Calvinh34 (calvinh34)]

Created attachment 83533
A gif showing a large width character bug in Konsole

[Calvinh34 (calvinh34)]

Hi !
Sorry, I did'nt get that the comment box in the web interface would be the message with my attachements.

Anyway, I found that bug too, using monofur font.

I also confirm it's font dependant, I have no problem with any other font like Deja Vu or Monospace.
Although I patched it with powerline () witch gives the arrow-like character in prompt, the problem actually come from the 'inverted A' and 'weird E'.

As you can see, the bug doesn't occur when the problematic glyph is the first character after prompt (the inverted A), which is just cropped to a normal width wharacter.

Arthur

[petteyg359 (petteyg359)]

(In reply to comment #7)
> I also confirm it's font dependant, I have no problem with any other font
> like Deja Vu or Monospace.

In my experience, this occurs with all fonts. Changing the font (either face or size) will solve the problem temporarily; after exiting and restarting Konsole, the problem appears again regardless of the current font.

[Iven Hsu (ivenvd)]

I confirm this issue.

In my case, the only working fonts are Dejavu and Monospace. All of the other fonts have this issue, like Droid Sans Mono, Ubuntu Mono, Adobe Source Code Pro, Fantasque Sans Mono.

While this doesn't occur in xterm.

[Cândido Lopes Fontes (canlotes)]

On my case is confirmed also with "Unispace" under openSUSE 13.1 x86-64 with fglrx64. It seems that the final character is under the cursor, no matter his shape or font size.

[Hexumg (hexumg)]

Created attachment 88421
konsole russian issue

[Hexumg (hexumg)]

Comment on attachment 88421
konsole russian issue

Issue: the cursor moves right from proper position when typing russian
Encoding: UTF8
Entered string: "при наборе русского текста курсор уходит вправо"
Font Oxygen Mono 8
Version 2.13.2

[Cândido Lopes Fontes (canlotes)]

I am starting to think that this is something related with the video driver (¿...? IMHO). I have an old computer, with a nvidia card, same initial installation but instead ati driver, it has the nvidia one. The same font type and it works perfectly.

[Eevee-kdebugs (eevee-kdebugs)]

I'm /reasonably/ sure this has nothing to do with Unicode character width, and everything to do with fonts. (Note that the character in the original screenshot doesn't take up two cells; it takes about one and a half.)

I think the problem is that Konsole is rendering each row as a regular line of text.

The original reporter's terminal font apparently doesn't have an airplane glyph. So the font engine searches for a font that does, and finds one, whatever it may be. But this new font probably isn't monospace, so the glyph isn't the same width as the rest of the characters in the terminal. Konsole blindly prints it anyway and places the next character immediately following it, even though this is now completely misaligned from the grid.

The cursor is apparently rendered separately (makes sense, since it's not a character), so it gets drawn in the appropriate grid cell.

What Konsole /should/ be doing is either shrinking the substituted character to fit in a cell, or forcibly rendering every single character at its correct position (even if this makes some overlap).

This has been a bug for as long as I've used KDE. It makes using non-ASCII characters for any reason basically impossible — my prompt, irssi, vim, roguelikes, anything that uses interesting characters causes the cursor and columns to become misaligned. I've had to switch to a libvte-based terminal, which correctly handles this by overlapping characters when necessary.

[Hexumg (hexumg)]

(In reply to Eevee from comment #14)
> I'm /reasonably/ sure this has nothing to do with Unicode character width,
> and everything to do with fonts. (Note that the character in the original
> screenshot doesn't take up two cells; it takes about one and a half.)

You are right. My problems ended when I set Droid font.
I think Konsole should warn user about non-monospace characters in selected font or try to calculate right cursor position. Something should be done to eliminate users confusion.

[Matthew Paul Thomas (mpt)]

Hi, I'm QA for the Ubuntu font featured in the screenshot. I don't use KDE, so take anything I say with a grain of salt, ;-) but:

> What Konsole /should/ be doing is either shrinking the substituted character to fit in a cell,

It would be inefficient and unreliable if the developers of every terminal app, and every other app that uses a monospace font for alignment reasons, had to include their own code to detect and shrink over-wide characters. It wouldn't even occur to most of them that it was something they needed to do! So in <http://launchpad.net/bugs/932958> I suggested that this be done lower down in the stack -- perhaps in fontconfig, perhaps in Pango (or whatever KDE's equivalent of Pango is).

> or forcibly rendering every single character at its correct position (even if this makes some overlap).

This is what Gnome Terminal does right now, and people really don't like the overlap. Examples:
http://launchpad.net/bugs/1116428
http://launchpad.net/bugs/1094434
http://launchpad.net/bugs/736613

[1-alexander-t (1-alexander-t)]

Created attachment 94081
Screenshot showing where the letters are cut off

Someone posted a new bug (bug 351380) yesterday which is a duplicate of this one, but it made it (in my opinion) more clear where the cutting off of the characters happens.

It was already mentioned that following a wide character, from the cursor on the rest is drawn correctly again, but this also happens on any other color change (and I could imagine maybe bold/underlined too, but I didn't check).
To show this, I expanded the test case from that bug and made a screenshot of it.
As you can see, the last misaligned character is always cut off when the color changes.
If you print only the text in red, it is cut off at the end of the line; if you don't change the color, you don't see any cut off, but if you just add one "red" space after the text, it will cut off the last character again.
This also happens when you select part of the line (presumably the fg/bg colors are just inverted for that part) or put the cursor in the line (but that isn't shown in the screenshot).
You can also see this happening in the input lines due to my syntax highlighting.

Tested with konsole 15.08.0

[Kurt Hindenburg (kurt-hindenburg)]

*** Bug 351380 has been marked as a duplicate of this bug. ***

[Brendon Higgins (blhiggins)]

Perhaps not surprising, but it seems vertical extent/placement is also affected by this. I was running a script (in Konsole 15.08.0) which prints names of files it's processing, and noticed some lines that happened to contain a Я (Cyrillic capital Ya) character in the filename were misplaced downwards by a couple of pixels when using the Oxygen Mono font (9pt).

[Serhiy (xintx-ua)]

Filed a report against Oxygen Mono: https://bugs.kde.org/show_bug.cgi?id=354588