fc-match Ryumin should return Mincho(serif) font in ja locale

[Nobuto Murata]

Ryumin and Gothic-BBB are generic font names for serif/sans-serif like Arial/Times new roman. fonts-takao-mincho has a rule[1] to offer substitute for Ryumin, however `fc-match Ryumin` does not return fonts-takao-mincho(serif), it returns fonts-takao-gothic(sans-serif) instead.

Expected result:
$ fc-match Ryumin
fonts-japanese-mincho.ttf: "TakaoPMincho" "Regular"

Actual result:
$ fc-match Ryumin
fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

[1]
/etc/fonts/conf.d/65-fonts-takao-mincho.conf
     21 <match target="pattern">
     22 <test qual="any" name="family">
     23 <string>Ryumin</string>
     24 </test>
     25 <edit name="family" mode="prepend">
     26 <string>Takao P明朝</string>
     27 </edit>
     28 </match>

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: fonts-takao-mincho 003.02.01-9ubuntu2
ProcVersionSignature: Ubuntu 3.19.0-7.7-generic 3.19.0
Uname: Linux 3.19.0-7-generic x86_64
ApportVersion: 2.16.2-0ubuntu1
Architecture: amd64
CurrentDesktop: Unity
Date: Tue Mar 10 22:41:44 2015
EcryptfsInUse: Yes
InstallationDate: Installed on 2015-02-28 (9 days ago)
InstallationMedia: Ubuntu 15.04 "Vivid Vervet" - Alpha amd64 (20150228)
PackageArchitecture: all
SourcePackage: fonts-takao
UpgradeStatus: No upgrade log present (probably fresh install)

[Nobuto Murata (nobuto)]

Example PDF document.

[Nobuto Murata (nobuto)]

The PDF document above is a template of tax declaration paper from Japanese National Tax Agency. It specifies both Ryumin(serif) and Gothic-BBB(sans-serif), but Evince uses one sans-serif font "TakaoPGothic" to substitute both.

[Nobuto Murata (nobuto)]

hm, it happens only with Japanese locale.

$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "TakaoPMincho" "Regular"

$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

[Nobuto Murata (nobuto)]

ok, it turned out that 65-fonts-takao-pgothic.conf is too strong. We need to figure out good balance between 65-fonts-takao-pgothic.conf and 65-fonts-takao-mincho.conf to return the correct substituted font for Ryumin in ja locale.

[Nobuto Murata (nobuto)]

lp:ubuntu/fonts-takao seems out-of-date unfortunately. I'm attaching debdiff instead.

[Nobuto Murata (nobuto)]

I don't think it causes any side effects for other locale. Because all binding="strong" are guarded by ja locale or ja specific font. But to be safe, adding CJK testers as subscriber to notify this change beforehand.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Rolf Leggewie (r0lf)]

Nobuto, thank you very much for your work on making Ubuntu support Japanese fonts properly. I can always learn so much in this respect from you and share your passion for visual aesthetics here.

As you may know, I usually remain with the LTS releases and only run the development release from time to time. As such, I hope the fix for this ticket will be backported to trusty as well. Furthermore, my system is mainly English-based. I hope you won't mind me adding my own experiences here.

I can basically confirm your findings, except that my US-locale returns a different font.

$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
ipamp.ttf: "IPAPMincho" "Regular"
$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

I tried the test PDF in different viewers and noticed there were some alignment issues in the viewer built into Firefox (please refer to my screenshot). I suspect that is another, separate issue?

One of the things I sometimes notice is that Chinese at times seems to have precedence over Japanese. For someone like me, that is unfortunate. Are you aware of a list of known issues or some website to check if one's configuration has any errors? This area of Ubuntu seems to change from release to release so it is hard to keep up (with a lot of improvement being made, at least that has been my experience). I do not know as much as you do, so at best, I will notice the effects but I would be utterly unable to dig into the causes. That's why your work is so much appreciated.

[Rolf Leggewie (r0lf)]

You titled the ticket " fc-match Ryumin should return Mincho(serif) font in ja locale". That name is so obviously Japanese that I wonder why the returned value should be locale-specific. From the superficial information I have it seems that optimizations for Japanese are sometimes only made only for Japanese locale, leaving people like me who only display and write Japanese in a non-Japanese locale out in the rain. I understand that sometimes this might have to do with political sensitivities and the power of numbers.

[Rolf Leggewie (r0lf)]

After applying your patches I can confirm that fc-match now returns a Mincho font.

$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "Takao P明朝" "Regular"
$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "TakaoPMincho" "Regular"

I cannot see any visual change in the test file in evince, though. I will have to admit I would not be good at spotting any difference, though. What words should now display in the more brush-like Mincho font?

[Nobuto Murata (nobuto)]

@Rolf,

> As you may know, I usually remain with the LTS releases and only run the development release from time to time. As such, I hope the fix for this ticket will be backported to trusty as well.

If this fix lands into vivid successfully and no regression reported in vivid, we can consider to backport it to trusty as well. However I'm under the impression that this Ryumin thing is not so critical.

> Furthermore, my system is mainly English-based. I hope you won't mind me adding my own experiences here.
>
> I can basically confirm your findings, except that my US-locale returns a different font.
>
> $ env LC_ALL=en_US.UTF-8 fc-match Ryumin
> ipamp.ttf: "IPAPMincho" "Regular"
> $ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
> fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

My patch fixes this behavior, the result for Ryumin will be always the same in all locales.

> I tried the test PDF in different viewers and noticed there were some alignment issues in the viewer built into Firefox (please refer to my screenshot). I suspect that is another, separate issue?

Yes, that's separate issue in Mozilla's pdf.js I think. The PDF I attached is rather complicated, I think it's natural rendering is broken in some viewers.

> One of the things I sometimes notice is that Chinese at times seems to have precedence over Japanese. For someone like me, that is unfortunate. Are you aware of a list of known issues or some website to check if one's configuration has any errors? This area of Ubuntu seems to change from release to release so it is hard to keep up (with a lot of improvement being made, at least that has been my experience). I do not know as much as you do, so at best, I will notice the effects but I would be utterly unable to dig into the causes. That's why your work is so much appreciated.

> You titled the ticket " fc-match Ryumin should return Mincho(serif) font in ja locale". That name is so obviously Japanese that I wonder why the returned value should be locale-specific. From the superficial information I have it seems that optimizations for Japanese are sometimes only made only for Japanese locale, leaving people like me who only display and write Japanese in a non-Japanese locale out in the rain. I understand that sometimes this might have to do with political sensitivities and the power of numbers.

Nowadays Ubuntu switches font substitutions by locales. For example, `fc-match serif` returns "DejaVu Serif" for most locales, "TakaoPMincho" for ja locale. You can specify per-user font locale settings(actually it's LC_CTYPE) in ~/.pam_environment or ~/.profile.
https://help.ubuntu.com/community/EnvironmentVariables#Session-wide_environment_variables

Can you please put "LC_CTYPE=ja_JP.UTF-8" in your ~/.pam_environment, re-login and see how it works?

[Nobuto Murata (nobuto)]

> I cannot see any visual change in the test file in evince, though. I will have to admit I would not be good at spotting any difference, though. What words should now display in the more brush-like Mincho font?

Right. I thought Ryumin was used in the doc more often, but it wasn't. We need to prepare more proper sample doc if we try SRU ;)

[Mitsuya Shibata (cosmos-door)]

> We need to prepare more proper sample doc if we try SRU ;)

Attached more simple sample.

[Sebastien Bacher (seb128)]

Thanks, I've sponsored the change to vivid, let's see what the feedback is there

[Launchpad Janitor (janitor)]

This bug was fixed in the package fonts-takao - 003.02.01-9ubuntu3

---------------
fonts-takao (003.02.01-9ubuntu3) vivid; urgency=medium

  * 65-fonts-takao-gothic.conf, 65-fonts-takao-pgothic.conf,
    65-fonts-takao-mincho.conf:
    - Use generic font names instead of localized names for readability
    - Add 'binding="strong"' inside all sections guarded by locale for
      consistency
    - Add 'binding="strong"' for ja specific fonts "GothicBBB" and
      "Ryumin" to substitute those fonts reliably (LP: #1430339)
 -- Nobuto Murata <email address hidden> Wed, 11 Mar 2015 01:45:37 +0900

[Mitsuya Shibata (cosmos-door)]

Just for later reference, attached display sample.