🏁 Spaces after some unicode characters are looking like unicode placeholder characters

[Removed by request]

I'm using Ubuntu 15.10 dev with unifont 1:8.0.01-1 and adding spaces after some unicode characters will cause them to look like unicode placeholder characters (a square with some characters in it). An example would be the 2 byte unicode character "D83C DFC1" which is drawn as a flag and looks this: 🏁
If I'm adding a space after the flag it looks this: 🏁
But it seems some browsers are drawing the space correctly but on system components the unicode placeholder character is shown. For this reason I have also added the flag and a space at the beginning of the summary to cause it to be shown in the title of the window manager.

[Paul Hardy (unifoundry)]

I don't think this is a bug with Unifont, but with a font rendering engine. I am on the road now and just have Debian Jessie with me using Unifont8.0.01-1. I don't have Ubuntu with me but my Debian setup should be close. The Iceweasel browser on my Debian Jessie system displays this bug report just fine. I've also tried this in Firefox with the same results. In both browsers I set the default font to Unifont.

The character you mention is in an upper Unicode plane, so it is not in the "Unifont" font; it is in the "Unifont Upper" font. Both TTF files are included in the "unifont" package. If you set your default font to Unifont though, I doubt that upper-plane glyphs are being taken from "Unifont Upper" unless you specifically tell the application you are running to do that.

TrueType has a hard limit of 2^16 glyphs per font, so it was necessary to put glyphs above Unicode Plane 0 (the Basic Multilingual Plane) in a separate font file.

If you are specifying upper-plane Unicode glyphs using Unicode surrogate pairs (U+D83C, U+DFC1), that might cause problems with an application expecting UTF-8. But if you saw the flag glyph (U+01F3C1), you must have entered it correctly in your original file.

I think this bug needs to be reassigned to whatever font rendering system is involved.

[Paul Hardy (unifoundry)]

Sworddragon,

Here is some more detail. The UTF-8 equivalent for the surrogate pair you entered should be (in octal) \360\237\217\201 in the web page you were viewing.

The Unifont package is split between a base "Unifont" font, which only contains Plane 0 glyphs (and the Copyleft glyph as of Unifont 11.0.01), and a "Unifont Upper" font, which contains glyphs above Plane 0. The TrueType format has a hard limit of 64k glyphs in one font, so the split into two fonts is necessary.

If you were using the "Unifont Upper" font rather than the base "Unifont" font to try to display that web page, I added a space glyph to it for the Unifont Upper 11.0.01 release. That is the only thing that might have caused that behavior that changing the TrueType font would influence. I could not reproduce the issue you saw even before the Unifont 11.0.01 release though.

As a data point for the font rendering engine, I found out last year that Harfbuzz uses its own determination for spacing of TrueType glyphs. It ignores x-axis offset information that the TrueType font contains. Harfbuzz only uses horizontal offset information in OpenType fonts, not in TrueType fonts. You can see the exchange related to that here:

     https://bugzilla.gnome.org/show_bug.cgi?id=787284

The end result of that should be that at least for combining characters, vertical placement will be improved but horizontal placement could be less than ideal (even incorrect, as observed in that GNOME bug report).

Adding the space glyph to "Unifont Upper" in version 11.0.01 was the only change I could make to a TrueType font that I did not already make. All horizontal offset information was already in all the TrueType fonts in the Unifont package. If this is still a problem, I do not see any way it could be from the TrueType fonts; it would have to be with the font rendering engine.

Thank you,

Paul Hardy

[Removed by request (removed3425744)]

On testing this with Ubuntu 18.10 dev with unifont 1:11.0.01-1 I do not see this issue anymore on the flag at this site (the browser document, the tab title, the window title and the taskbar title are all fine). I also did test this on other sites with other characters where I did usually see this issue and I couldn't reproduce it there too.