Configuration of the Droid Sans Fallback font

Bug #1351092 reported by Gunnar Hjalmarsson on 2014-07-31
26
This bug affects 2 people
Affects Status Importance Assigned to Milestone
fonts-android (Debian)
Fix Released
Unknown
fonts-android (Ubuntu)
Medium
Gunnar Hjalmarsson
Trusty
Medium
Gunnar Hjalmarsson
language-selector (Ubuntu)
Medium
Gunnar Hjalmarsson
Trusty
Medium
Gunnar Hjalmarsson

Bug Description

[Impact]

fontconfig configuration settings, aimed at making the Droid Sans Fallback font be used to render Chinese, fails sometimes in an unpredictable way. This seems to be caused by the file 65-droid-sans-fonts.conf, which is currently installed by fonts-droid. Consequently that file is proposed to be dropped from fonts-droid. Some fontconfig files in language-selector are proposed to be changed accordingly.

Example problems:

* One of the AR PL UMing fonts is sometimes used instead of Droid Sans
  Fallback in case of a Chinese locale.

* Buggy rendering of Chinese contents in qt apps in case of a
  non-Chinese locale (bug #1334495).

* Droid Sans Fallback can't be used in Ubuntu Touch (see discussion at
  bug #1346766).

[Test Case]

Since the behaviour is not always buggy, it's hard to present a proper test case. Instead I have to refer to the discussions in this bug report as well as the above mentioned bugs.

[Regression Potential]

While the fonts-droid package installs a bunch of fonts for various languages, only the Droid Sans Fallback font is used as part of Ubuntu's default font configuration. The regression risk for Chinese users is reasonably very low. There is a risk, though, that this change leads to surprise changes for individual users who make use of other fonts but Droid Sans Fallback. There is no indication that those other fonts are widely used.

[Original description]

There are currently several open issues related to the use of Droid Sans Fallback for rendering Chinese content:

* Two mixed fonts when rendering Chinese in KDE/QT apps with Droid
  Sans fonts
  https://launchpad.net/bugs/1334495

* Droid Sans no longer preferred font for Chinese
  https://launchpad.net/bugs/1335482

* Chinese in Ubuntu Touch should use Heiti style sans serif font
  https://launchpad.net/bugs/1346766

Unlike e.g. fonts-wqy-microhei, the fonts-droid package installs a bunch of fonts, of which only one is needed for Chinese. In an attempt to sort things out I have built the fonts-android source package in my PPA with the DroidSansFallbackFull.ttf font broken out to a separate binary package named fonts-droid-cjk. The PPA also includes a version of language-selector where the changes in version 0.129.2 have been reverted.

To test it in Trusty, you should:

* Uninstall the fonts-droid package

* Install fonts-droid-cjk and language-selector-common from my PPA
  at https://launchpad.net/~gunnarhj/+archive/ubuntu/droid-test

My own tests indicate that the change to language-selector due to bug #1335482 was a step in the wrong direction. With DroidSansFallbackFull.ttf as the only installed font from the Droid Sans family, you get rid of possible confusion that might have resulted in the issue reported in that bug.

$ LANG=zh_CN.UTF-8 fc-match -s 'sans-serif' | head -n 5
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
uming.ttc: "AR PL UMing CN" "Light"
uming.ttc: "AR PL UMing HK" "Light"
ukai.ttc: "AR PL UKai CN" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"

Also, if we would take this route, it might be easier to fix a configuration that makes Droid Sans Fallback work well with qt apps. (This is pure theory/hope so far.)

Looking forward to your comments.

Lukas Bunsen (lukasbunsen) wrote :

Seems to work for me, after I installed the two packages all characters in Chinese text are displayed correctly in KDE applications, for example Kate, only using Droid.

The output of "locale" is mixed again, I am not sure if that is due to the update of language selector or because I tried several languages before to try different locales. Same goes for the language actually used, it is an weird mix of English and German even within the same application, for example the Kickoff-Menu.
/etc/default/locale is only set to LANG=de_DE.UTF-8, so I don't really know why I have that new issue, or how to change it.

 As part of that, running locale gives "locale: Cannot set LC_ALL to default locale: No such file or directory" (in German) on top of the normal output lines, same when installing new packages.

"locale: Kann LC_ALL nicht auf die Standard-Lokale einstellen: Datei oder Verzeichnis nicht gefunden
LANG=de_DE.UTF-8
LANGUAGE=de:en:zh:en
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC=de_GB.UTF-8
LC_TIME=de_GB.UTF-8
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY=de_GB.UTF-8
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER=de_GB.UTF-8
LC_NAME=de_GB.UTF-8
LC_ADDRESS=de_GB.UTF-8
LC_TELEPHONE=de_GB.UTF-8
LC_MEASUREMENT=de_GB.UTF-8
LC_IDENTIFICATION=de_GB.UTF-8
LC_ALL="

Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-08-01 07:40, Lukas Bunsen wrote:
> Seems to work for me, after I installed the two packages all
> characters in Chinese text are displayed correctly in KDE
> applications, for example Kate, only using Droid.

Without further config tweaking? Yes!! :) Finally some progress.

@Rex: Even if fonts-wqy-microhei just was added to the touch seed, it would be valuable to know if fonts-droid-cjk would work on the phone. Could you please test and let us know?

So this seems to be all about packaging/configuration. The question is what to do next...

(@Lukas: As regards the locale, I think the Kubuntu region GUI allows you to set language and country separately. This has resulted in the non-existing locale name "de_GB.UTF-8". Try to find Great Britain and replace it with Germany.)

Cheng-Chia Tseng (zerng07) wrote :

OK, after I removed fonts-droid package and installed those testing packages and configurations which using "Droid Sans Fallback" instead of "Droid Sans" today, everything seems to be good. Below are my testing results,

LANG=zh_TW.UTF-8 fc-match -s "Droid Sans" | head -n 10
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
ukai.ttc: "AR PL UKai TW" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
Waree.ttf: "Waree" "Book"
ukai.ttc: "AR PL UKai CN" "Book"
uming.ttc: "AR PL UMing CN" "Light"
KhmerOS.ttf: "Khmer OS" "Regular"

LANG=zh_TW.UTF-8 fc-match -s "sans-serif" | head -n 10
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
ukai.ttc: "AR PL UKai TW" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
Waree.ttf: "Waree" "Book"
ukai.ttc: "AR PL UKai CN" "Book"
uming.ttc: "AR PL UMing CN" "Light"
KhmerOS.ttf: "Khmer OS" "Regular"

LANG=zh_TW.UTF-8 fc-match -s "Droid Sans Fallback" | head -n 10
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
ukai.ttc: "AR PL UKai TW" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
Waree.ttf: "Waree" "Book"
ukai.ttc: "AR PL UKai CN" "Book"
uming.ttc: "AR PL UMing CN" "Light"
KhmerOS.ttf: "Khmer OS" "Regular"

Cheng-Chia Tseng (zerng07) wrote :

I don't know why the configuration provided by fonts-droid package makes fontconfig match UMing first as "Droid Sans Fallback". That is so weird. Howver, removing fonts-droid eliminates the problem succesefully.

Gunnar Hjalmarsson (gunnarhj) wrote :

Yes, it's weird. Right now I suspect that the configuration file 65-droid-sans-fonts.conf (attached), which the current fonts-droid package installs, is the root cause of this mess. That file seems to imply that if you install fonts-droid, you want Droid Sans fonts be used for all languages, while we are currently only interested in Droid Sans Fallback. In other words, it does not fit well with Ubuntu's default font configuration.

I have modified the packages in my PPA a little. From the changelogs:

fonts-android (1:4.3-3ubuntu1.1~ppa2) trusty; urgency=medium

  * Binary package split:
    - fonts-droid split into fonts-droid-cjk, which installs the Droid
      Sans Fallback font, and fonts-droid-general, which installs the
      rest. fonts-droid converted to a transitional dummy package
      which depends on fonts-droid-cjk.
  * debian/local/65-droid-sans-fonts.conf:
    - Recipes affecting Droid Sans Fallback dropped.

language-selector (0.129.3~ppa2) trusty; urgency=low

  * fontconfig/69-language-selector-zh-??.conf:
    Reverse changes in version 0.129.2.
  * data/pkg_depends:
    Replace fonts-droid with fonts-droid-cjk.

65-droid-sans-fonts.conf is installed by the new fonts-droid-general package, but the Droid Sans Fallback related entries have been removed. I think that the packages in my PPA are now in a state which makes them uploadable to the archive.

You may ask: Do we really need this package split? Wouldn't it be sufficient to modify (or drop) 65-droid-sans-fonts.conf? Honestly I'm not sure. To test that theory, you can simply install both fonts-droid-cjk and fonts-droid-general from my ppa. When I do so on my 14.04 desktop, it seems to work. At least for the moment...

@Lukas: Can you please do the same, and let us know if Chinese is still rendered correctly in the qt apps?

OTOH: Is it sensible to have to install all those fonts just to be able to use Droid Sans Fallback for Chinese? And what about the phone?

@Iain: Your comments on all this would be appreciated. Apparently something needs to be done, also in trusty.

Iain Lane (laney) wrote :

I'll poke, but I can't get even the "bad" output on my current Utopic system -

laney@iota> LANG=zh_TW.UTF-8 fc-match -s sans-serif | head -5
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
Verdana.ttf: "Verdana" "Normal"

I do have the locale installed. How do you guys get this? appending :lang=zh-cn does get me droid first.

Changed in fonts-android (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Changed in language-selector (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Changed in fonts-android (Ubuntu Trusty):
importance: Undecided → Medium
status: New → Triaged
Changed in language-selector (Ubuntu Trusty):
importance: Undecided → Medium
status: New → Triaged
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-08-04 12:24, Iain Lane wrote:> laney@iota
> LANG=zh_TW.UTF-8 fc-match -s sans-serif | head -5
> DejaVuSans.ttf: "DejaVu Sans" "Book"
> DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
> DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
> DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
> Verdana.ttf: "Verdana" "Normal"

Hmm.. That would indicate that zh_TW.UTF-8 is not generated on your system.

Gunnar Hjalmarsson (gunnarhj) wrote :

Aha, just saw that the locale is installed. Well, then take it as an example of the current unreliable behaviour.

Iain Lane (laney) wrote :

It was because I had $LANGUAGE set. Now it's--

laney@iota> LANG=zh_TW.UTF-8 fc-match -s sans-serif | head -5
DroidSansFallbackFull.ttf: "Droid Sans" "Regular"
DroidSans.ttf: "Droid Sans" "Regular"
DroidNaskh-Regular.ttf: "Droid Sans" "Regular"
DroidSansEthiopic-Regular.ttf: "Droid Sans" "Regular"
DroidSansHebrew-Regular.ttf: "Droid Sans" "Regular"

is that right? If not, what should it be?

Gunnar Hjalmarsson (gunnarhj) wrote :

Well, you get DroidSansFallbackFull.ttf first, which is good. That's what I got previously too (see comment #10 of bug #1335482), and it was the intention with

https://launchpad.net/ubuntu/+source/language-selector/0.129.2

However, after having tried various things back and forth, I don't get that result any longer. Instead I get:

$ dpkg-query -W fonts-droid language-selector-common
fonts-droid 1:4.3-3ubuntu1
language-selector-common 0.129.2
$ LANG=zh_TW.UTF-8 fc-match -s sans-serif | head -5
DroidSans.ttf: "Droid Sans" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
ukai.ttc: "AR PL UKai TW" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"

I.e. approximately the same result as Cheng-Chia Tseng reported in comment #9 of bug #1335482. And that's not good, since it leads to "AR PL UMing TW" being used to render traditional Chinese contents instead of "Droid Sans Fallback".

So the result of the current configuration is unpredictable, and I'm pretty sure by now that the culprit which causes this unpredictability is 65-droid-sans-fonts.conf. The pending merge proposals should fix that.

I also think that the merge proposals will both fix bug #1334495 and make it possible to use fonts-droid in the phone.

Lukas Bunsen (lukasbunsen) wrote :

I installed both onts-droid-cjk and fonts-droid-general, Chinese text is still displayed correctly.

description: updated
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks for reporting that, Lukas. Much appreciated. It confirms that the latest proposed changes should be sufficient to fix bug #1334495 too.

Changed in fonts-android (Debian):
status: Unknown → New
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fonts-android - 1:4.3-3ubuntu2

---------------
fonts-android (1:4.3-3ubuntu2) utopic; urgency=medium

  [ Gunnar Hjalmarsson ]
  * debian/local/65-droid-sans-fonts.conf:
    - Dropped. We want to cherry pick Droid Sans Fallback for rendering
      Chinese content, and 65-droid-sans-fonts.conf has defeated that
      purpose and caused confusion (LP: #1334495, LP: #1351092).

  [ Iain Lane ]
  * Add fonts-droid.maintscript to clean up the now obsolete conffile on
    upgrade.
 -- Gunnar Hjalmarsson <email address hidden> Mon, 15 Sep 2014 14:56:33 +0100

Changed in fonts-android (Ubuntu):
status: In Progress → Fix Released
Iain Lane (laney) on 2014-09-15
Changed in fonts-android (Ubuntu Trusty):
status: Triaged → In Progress
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
Changed in fonts-android (Ubuntu Trusty):
status: In Progress → Fix Committed
Changed in language-selector (Ubuntu Trusty):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
status: Triaged → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package language-selector - 0.133

---------------
language-selector (0.133) utopic; urgency=low

  * fontconfig/69-language-selector-zh-??.conf:
    Reverse changes in version 0.132 (LP: #1334495, LP: #1351092).
 -- Gunnar Hjalmarsson <email address hidden> Mon, 15 Sep 2014 16:23:00 +0200

Changed in language-selector (Ubuntu):
status: In Progress → Fix Released
Gunnar Hjalmarsson (gunnarhj) wrote :

@Lukas: These issues are about to be fixed now. The solution, which is about to be uploaded to both trusty and utopic, does not include a split of the fonts-droid package. Consequently you should uninstall the fonts-droid-cjk and fonts-droid-general packages from my PPA, to not confuse your system. You should also remove the ~/.config/fontconfig/conf.d/65-droid-sans-first.conf (from the bug #1334495 discussion) if you haven't already.

Hello Gunnar, or anyone else affected,

Accepted language-selector into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/language-selector/0.129.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed
Brian Murray (brian-murray) wrote :

Hello Gunnar, or anyone else affected,

Accepted fonts-android into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/fonts-android/1:4.3-3ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Gunnar Hjalmarsson (gunnarhj) wrote :

I have installed fonts-droid 1:4.3-3ubuntu1.1 and language-selector-common 0.129.3 from trusty-proposed. Droid Sans Fallback is the preferred font in case of a Chinese locale:

$ LANG=zh_CN.UTF-8 fc-match 'sans-serif'
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"

In case of a non-Chinese locale, Droid Sans Fallback is the first Chinese font in the list:

$ LANG=en_US.UTF-8 fc-match -s 'sans-serif' | head -12
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
Verdana.ttf: "Verdana" "Normal"
Arial.ttf: "Arial" "Normal"
n019003l.pfb: "Nimbus Sans L" "Regular"
Waree.ttf: "Waree" "Book"
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
ukai.ttc: "AR PL UKai CN" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
uming.ttc: "AR PL UMing CN" "Light"

tags: added: verification-done
removed: verification-needed
Lukas Bunsen (lukasbunsen) wrote :

Thanks for the hint, Gunnar.
I installed the package from proposed and rebooted, Chinese text still is displayed correctly, thus the bug solved for me.

Gunnar Hjalmarsson (gunnarhj) wrote :

Splendid!

I just deleted ppa:gunnarhj/droid-test, since it now has served its purpose.

Changed in fonts-android (Debian):
status: New → Unknown
Jackie (jackie-space) wrote :

Hi,

I had this issue with Chrome/Chromium and Skype.

After installing fonts-droid from trusty-proposed solved the issue.

version installed: fonts-droid (1:4.3-3ubuntu1.1)

Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-09-19 23:35, Jackie wrote:
> I had this issue with Chrome/Chromium and Skype.
>
> After installing fonts-droid from trusty-proposed solved the issue.

Yeah, dropping 65-droid-sans-fonts.conf seems to solve a Chrome/Chomium issue as well.

https://code.google.com/p/chromium/issues/detail?id=316723

And now you say Skype too? Excellent! :)

Jackie (jackie-space) wrote :

Yes, Skype is working magically as well! Thanks!

Changed in fonts-android (Debian):
status: Unknown → New
Changed in fonts-android (Debian):
status: New → Fix Released
Megan Gilbert (megilbert) wrote :

This fix worked for me, too. Thank you!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package language-selector - 0.129.3

---------------
language-selector (0.129.3) trusty-proposed; urgency=low

  * fontconfig/69-language-selector-zh-??.conf:
    Reverse changes in version 0.129.2 (LP: #1334495, LP: #1351092).
 -- Gunnar Hjalmarsson <email address hidden> Mon, 15 Sep 2014 16:34:00 +0200

Changed in language-selector (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for language-selector has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fonts-android - 1:4.3-3ubuntu1.1

---------------
fonts-android (1:4.3-3ubuntu1.1) trusty; urgency=medium

  [ Gunnar Hjalmarsson ]
  * debian/local/65-droid-sans-fonts.conf:
    - Dropped. We want to cherry pick Droid Sans Fallback for rendering
      Chinese content, and 65-droid-sans-fonts.conf has defeated that
      purpose and caused confusion (LP: #1334495, LP: #1351092).

  [ Iain Lane ]
  * Add fonts-droid.maintscript to clean up the now obsolete conffile on
    upgrade.
 -- Gunnar Hjalmarsson <email address hidden> Mon, 15 Sep 2014 14:56:33 +0100

Changed in fonts-android (Ubuntu Trusty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.