Unicode combining characters don't combine properly.

[Adam Buchbinder]

Binary package hint: firefox

Load this Unicode test page:

http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt

Scroll down to "Combining characters". The accents will appear to the right of the characters they're meant to appear directly over.

I'm running firefox 1.5.dfsg+1.5.0.4-0ubuntu6.06 on Ubuntu Dapper.

[Adam Buchbinder (adam-buchbinder)]

This is how the above-mentioned test file appears in my copy of firefox. The combining accents should be above the characters, but they appear to the right of them.

[Daniel Robitaille (robitaille)]

It looks the same on my laptop.

[David Farning (dfarning)]

Still appears to be an issue with firefox 2.0

David

[Alexander Sack (asac)]

Adam, looking at your screenshot you appear to lack some fonts. Do you still see the problem? What firefrox version are you on? Do you have a screenshot of how it should look properly?

Thanks,

 - Alexander

[Adam Buchbinder (adam-buchbinder)]

I'm running firefox 1.5.dfsg+1.5.0.11-0ubuntu0.6.06.1, the current version in Dapper with security and updates. Yes, it's still doing it, as nothing has changed. I see that I'm missing a font containing U+20D1 (COMBINING RIGHT HARPOON ABOVE), and that a placeholder character is being displayed there; this is not a bug. The bug report is that the combining characters that *do* exist (the over-ring above the lambda, the dot and double-dot above v and x) aren't combining. If you have a screenshot of this page rendering on your system that places the ring over the lambda and the dots over the v and the x, please attach it and specify your version of firefox and other libraries which might be involved.

I'm running libfreetype6 2.1.10-1ubuntu2.3 and libpango1.0-0 1.12.3-0ubuntu3; if there are other packages which influence font rendering, let me know and I'll attach that information as well. If this is a problem in a font rendering library, let me know which one and (hopefully) send me to a test program which simply displays characters in that font. If this is a bug in a font, which doesn't properly specify combining-character information, please let me know how to find out which font it is, and who to send a bug report to in that case.

[Adam Buchbinder (adam-buchbinder)]

Provided requested information; if there are other questions I can answer, please set back to 'Needs Info'.

[Alexander Sack (asac)]

please don't confirm your own bugs.

So you can't get a proper screenshot somewhere? Or reduce the testcase to just those letters that are broken?

[Adam Buchbinder (adam-buchbinder)]

I didn't confirm my own bug; Daniel Robitaille confirmed this bug ten months ago. I set it from 'Needs Info' back to 'Confirmed' because I answered the questions and figured that it no longer Needed Info. I don't have a "proper screenshot"; I don't have a browser that properly renders these, which is why I filed this bug.

These are the characters in question; I copied and pasted them from the text file linked in the initial bug report. They'll display properly (as three characters, each with a diacritical mark) in your browser, or they won't.

Λ̊ v̇ r̈

I've attached a screenshot of Firefox 1.5.dfsg+1.5.0.11-0ubuntu0.6.06.1 on Dapper, displaying this text both as fixed-width text in the main window, and as variable-width text in the quick-search bar at the upper right. You can see that the dots are above the letters v and r in the quick-search bar, and that the ring is almost above the capital lambda--it should be directly on top, but at least it's not a full character's width to the right like it is in the fixed-width font in the main window.

[Alexander Sack (asac)]

ok ... sorry for the bouncing. I set this bug to in progress now and it assign it to me, so it stays on top of my todo list.

[Alexander Sack (asac)]

Attached a screenshot taken from firefox-granparadiso alpha5 package in gutsy ... looks better?

[Adam Buchbinder (adam-buchbinder)]

Much better. The lambda (in the 'STARGATE' text) should have the ring directly over its center, as in Å/å, rather than slightly off to one side as it does now, but the dots over 'v' and 'r' are where they should be (as are the vector-arrows, but I don't know if those were broken in the first place). If that fix is released, the problem with the ring over the lambda can probably be downgraded from 'Medium' to 'Low'.

[dotancohen (dotancohen)]

> The lambda (in the 'STARGATE' text) should have the ring directly over its center, as in Å/å, rather than
> slightly off to one side as it does now

That seems due to the default font size having an odd number of pixels. Increasing the font size makes the issue disappear. It looks like this issue can be closed.

[Gustav Hartvigsson (gustav-hartvigsson)]

eehh, now the ring is over the T.
is this a regression or is it a new behaviour in UNICODE?

Browser:
Mozilla/5.0 (X11; U; Linux x86_64; sv-SE; rv:1.9.2.6pre) Gecko/20100610 Ubuntu/10.04 (lucid) Namoroka/3.6.6pre

[0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy (0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy)]

User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0b9pre) Gecko/20101219 Firefox/4.0b9pre
Build Identifier:

Does not display UTF-8 on the link, you can see an example of the text.
Affects the display of the operating system and browser.

Reproducible: Always

[Bugzilla-tf (bugzilla-tf)]

This is wfm using Mozilla/5.0 (Windows NT 6.1; rv:2.0b9pre) Gecko/20101217 Firefox/4.0b9pre SeaMonkey/2.1b2pre

>Affects the display of the operating system and browser.
Affects the display of the operating system ?
What do you mean with that ?

[Bzbarsky (bzbarsky)]

Worksforme as well. View menu says it's UTF-8, and all the non-ASCII bits show up fine. Reporter, can you please attach a screenshot of what you see?

[Kevin Brosnan (kbrosnan)]

works for me - Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b9pre) Gecko/20110103 Firefox/4.0b9pre

Server is sending the correct mime type. Possibly the user does not have international/symbol fonts installed that are needed for the page?

[0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy (0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy)]

Created attachment 501269
Screen

An article on this issue: http://habrahabr.ru/blogs/browsers/110331/
For me personally, in different browsers totally different. Well, firefox as you can see the screenshot incorrectly displays.

[0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy (0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy)]

Problems with language in a larger case no. Problems with:
Mathematics and stsientses
Nicer typography in plain text Phils
Combining charatsters
Thai (UCS Level 2)
Box drawing alignment

[0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy (0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy)]

Something like this should happen: http://img522.imageshack.us/img522/7060/1292858076615.пнж
According to people in the article, these problems are mainly in Windows, support for displaying Unicode-characters is not complete.

[0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy (0cs935kb517wwmwa7m9428daadkyev88fk76o0c6sq-9teq-wz6bkyhu4uqpfausw0ege9b0y33ege6o4wzvqwe1gy)]

http://img522.imageshack.us/img522/7060/1292858076615.png

[Kevin Brosnan (kbrosnan)]

I don't read Russian but the roughly translated article via Google translate seems to have some errors.

It seems to suggest that Firefox does not use curly apostrophes which is false. If you search the document for flat quotes the first such instance is "A Christmas Carol" and if you search for the curly quotes the characters are correctly matched.

Alignment issues are going to be limitations of the monospace fonts on the OS. If a character is not found in a monospace font and is found in a proportional font browsers will use the proportional font thus breaking the alignment.

All the issues in the page I see look to be from monospaced fonts for my OS (Win7) failing to have the glyphs required by the document.

[Mozaic (mozaic)]

Reproduce in Ubuntu 12.04

Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1

Combine correctly with Gedit 3.4.1

Similar bugs ??
https://bugzilla.mozilla.org/show_bug.cgi?id=197649

[Mozaic (mozaic)]

Bug in Launchapd:
https://bugs.launchpad.net/firefox/+bug/51554

[Adolfo Jayme Barrientos (fitojb)]

I can't reproduce this in Ubuntu 12.10. I'm inclined to think this was actually a bug in the font (DejaVu Sans Mono) and not in Firefox itself

[Vseerror (vseerror)]

(In reply to Kevin Brosnan from comment #8)
> All the issues in the page I see look to be from monospaced fonts for my OS
> (Win7) failing to have the glyphs required by the document.

So not a firefox bug?

Status of https://bugs.launchpad.net/firefox/+bug/51554 is invalid, with the comment "I can't reproduce this in Ubuntu 12.10. I'm inclined to think this was actually a bug in the font (DejaVu Sans Mono) and not in Firefox itself"

[Kevin Brosnan (kbrosnan)]

On Windows 7 today's Firefox nightly. This install of Windows7 does not have any additional fonts installed. As I recall my comment 3 I had a system with the STIX fonts installed.

Firefox will use a proportional glyph if a monosapced glyph does not exist on the system. This will cause the text document to have an odd alignment on rows where Unicode monospaced glyphs do not exist on the system.

I verified this assumption by opening the text document in Sublime text editor. It shows many instances where Windows does not have the correct monospaced glyph, it displays the Unicode missing glyph symbol rather than doing fallback.

I believe that doing fallback is the better thing to do for web browsers. Jonathan Kew would know for sure, or possibly there are older bugs to reference? If people want monospaced line drawing to work they should have a robust monospace font installed such as the ones provided by the STIX project.

http://sourceforge.net/projects/stixfonts/files/latest/download
https://en.wikipedia.org/wiki/STIX_Fonts_project
http://www.stixfonts.org/

[Joachim Durchholz (jo-durchholz)]

I'm seeing wrong combining behaviour on a Linux box with a monospaced font, so it's neither limited to Windows nor to variable-width fonts.

[Joachim Durchholz (jo-durchholz)]

More information:

I have:
Mozilla Firefox 38.0
Web page that displays this behaviour: https://github.com/cbm755/sympy/commit/cad689d3b82aa332a55b601d1a004f26fa6108f6 , search for "assert upretty( Symbol('Fhat') ) == u('F" (without the quotes) and the problem is directly after that.
CSS-requested font list: Consolas,"Liberation Mono",Menlo,Courier,monospace
Used font: Liberation Mono is used

Interestingly, the combining character is immediately followed by a span:
<span class="pl-s"><span class="pl-pds">'</span>F̂<span class="pl-pds">'</span></span>
Wild guess: the span right after Unicode COMBINING HAT disrupts Mozilla's rendering engine in some way. (It would explain why the issue is hard to reproduce.)
Detail: The pl-pds span changes font color, to a color of the same value; i.e. this may or may not contribute to the situation.

[Joachim Durchholz (jo-durchholz)]

I spoke too soon.
Same page, but look for "assert upretty( Symbol('xHATNorm') ) == u('‖" - the hat gets displayed over the vertical bars to the right of x, and there's no span change between x and vertical bars.

[Vseerror (vseerror)]

FYI, this reporter is gone - not responded to any bug reports. Perhaps this can be closed?