ttf-wqy-microhei ttf-wqy-zenhei break Korean fonts

[ahavatar]

Binary package hint: yelp

the default installation of Ubuntu 9.10 breaks Korean fonts because of the following packages

ttf-wqy-microhei
ttf-wqy-zenhei

They seemed to be installed with OpenOffice and overwrite the system default Korean fonts. But their Korean fonts are broken and almost unreadable.

I just removed the ttf-wqy-* packages and now the Korean fonts look great.

Please change the fonts priority order or fix broken fonts in the packages.

Thank you.

ProblemType: Bug
Architecture: amd64
Date: Thu Nov 5 19:46:49 2009
DistroRelease: Ubuntu 9.10
ExecutablePath: /usr/bin/yelp
InstallationMedia: Ubuntu 9.10 "Karmic Koala" - Release amd64 (20091027)
Package: yelp 2.28.0-0ubuntu2
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.48-generic
SourcePackage: yelp
Uname: Linux 2.6.31-14-generic x86_64

[Jordi Rovira i Bonet (jordi-rovira)]

I confirm this bug. I had to manually remove ttf-wqy-zenhei, because my korean fonts were unreadable.

[Qianqian Fang (fangq)]

I am not sure about what you mean by "breaks Korean fonts". If you are looking at Han characters under en_US.UTF8 locale, you really should not expect to see Korean Hanja, because Chinese, Japanese and Korean share those code points and (simplified) Chinese Han characters are used as the default (for non-CJK locales).

If you expect to use the font variants for Korean Hanja, then please set your locale to Korean, and do fontconfig-voodoo of your locale. in this case, cjk-specific fontconfig files will be activated to show your local font variants.

[ahavatar (kbn00)]

Even my local is en_US.UTF8, I should be able to visit some Korean websites, right? Without ttf-wqy-* packages, I have no problem in doing so with the Ubuntu 9.10 system default (i.e. I haven't changed any font nor locale setting except adding Korean language, but the default is still en_US.UTF8)

But with the ttf-wqy-* fonts installed, Korean fonts become broken and almost unreadable. In fact, many Korean Ubuntu 9.10 users have the same problem and have reported this in the Korean Ubuntu User forum (www.ubuntu.or.kr) as well.

[Qianqian Fang (fangq)]

@ahavatar:

Looks like you are talking about Hanguls, not the Hanjas. This does seems to be odd, as ttf-wqy-zenhei contains exactly the same Hangul glyphs as in the Un-fonts, which I believe is the default Korean font. Therefore, even ttf-wqy-zenhei overwrites unfont, it should not make any difference.

On my laptop, I am using 9.10 with the default en_US locale, I have no problems viewing the website in your post with both wqy fonts installed.

Can you attach a screenshot for better diagnosis of the problem?

[ahavatar (kbn00)]

It's not my posting, but there's a someone else posting with a screenshot on the Korean Ubuntu User forum.

The link is

http://www.ubuntu.or.kr/viewtopic.php?f=10&t=8232

In the screenshot, the Korean fonts' anti-aliasing is broken. Mine has the same problem with the ttf-wqy-* fonts installed. I hope this helps.

By the way, to reproduce this, 1) install Ubuntu 9.10 in English 2) add Korean language support without changing any setting.

[YunQiang Su (wzssyqa)]

it seem not so bad.
maybe wqy is not very well for Korean fonts,but we have no better choice.
maybe you can install a new font and use it.

the best way is to cotrubte wqy,it is a fonts project for cjk.

or you can give a better font that is free and display cjk good at same time.

[Qianqian Fang (fangq)]

@ahavatar:
can you do this test for me? install ttf-wqy-zenhei and restore the original settings (if you have made any additional changes), then, type

 sudo rm /etc/fonts/conf.d/66-wqy-zenhei-sharp*.conf

and then restart firefox and see if the problem still exists.

[ahavatar (kbn00)]

I reinstalled ttf-wqy-zenhei and it again breaks Korean(hangul) fonts, but after I removed the font.d conf file as you suggested, now Korean fonts look great again.

So, I guess something's wrong in the 66-wqy-zenhei-sharp*.conf files.

[Qianqian Fang (fangq)]

Here is what happened:
in 9.10, 66-wqy-zenhei-sharp.conf is linked under conf.d, which activates the embedded bitmaps in Zen Hei for CJK character display. Because Zen Hei does not have embedded bitmaps for Hangul glyphs (except 12pt), therefore, Hanguls are shown without AA (because setting antialiase=true is required to enable bitmaps).

here are two work-arounds:
1. unlink 66-wqy-zenhei-sharp*, which disable bitmaps for Chinese for all locales
2. add the following line in the first match-block in 66-wqy-zenhei-sharp.conf
        <test name="lang" compare="contains"><string>zh</string></test>
 this will continue using bitmaps for Chinese for zh_* locales, but disable it for other locales, including en_US

I am assigning zhengpeng, the Ubuntu maintainer for this package, to fix this bug.

@zhengpeng:
which way do you prefer?

[Qianqian Fang (fangq)]

a typo: setting antialiase=*false* is required to enable bitmaps

[Pan, SZ (pan.sz)]

For a long time when wqy was not present in ubuntu, Chinese text does
not look good in en_US.utf8. Because they are shown as Korean or Japan
font.

So, if this change is reversed, should Chinese users report the bugfix
itself as a bug?

The point is: when locale is en_US.utf8, only *one* of CJK fonts will look good.

If you put a Korean fonts as the highest priority, then Chinese and
Japan text will not look good at en_US.utf8.

If we put a Chinese fonts as the highest priority, then Korean and
Japan text will not look good at en_US.utf8.

If we put a Japan fonts as the highest priority, then Korean and
Chinese text will not look good at en_US.utf8.

The point is you should set the locale in order to raise your font to
the highest priority. Otherwise, in en_US.utf8 no one could decide
which one in CJK should be the highest priority.

Or there may be another solution: make a unique font which looks good
in all CJK characters...

On Fri, Nov 6, 2009 at 12:50 PM, ahavatar <email address hidden> wrote:
> Even my local is en_US.UTF8, I should be able to visit some Korean
> websites, right? Without ttf-wqy-* packages, I have no problem in doing
> so with the Ubuntu 9.10 system default (i.e. I haven't changed any font
> nor locale setting except adding Korean language, but the default is
> still en_US.UTF8)
>
> But with the ttf-wqy-* fonts installed, Korean fonts become broken and
> almost unreadable. In fact, many Korean Ubuntu 9.10 users have the same
> problem and have reported this in the Korean Ubuntu User forum
> (www.ubuntu.or.kr) as well.
>

[Arne Goetje (arnegoetje)]

Pan, Shi Zhu wrote:
> The point is you should set the locale in order to raise your font to
> the highest priority. Otherwise, in en_US.utf8 no one could decide
> which one in CJK should be the highest priority.

We have fontconfig-voodoo for this case as a workaround.
Language-selector contains fontconfig snippets for each region and the
command line script fontconfig-voodoo which puts one of those in place,
depending on the locale you give it as an argument.

See 'fontconfig-voodoo -h'.

> Or there may be another solution: make a unique font which looks good
> in all CJK characters...

That is highly desirable, but unfortunately simply impossible as long as
each region (China, Hong Kong, Taiwan, Japan, Korea) has their own
definition on how the glyphs should look like. The glyphs look different
in each region, but share the same codepoints. This has finally prompted
Unicode to publish the Code charts regarding CJK as multi-column tables,
which list all the different shapes [1].
Therefor it's technically impossible.

[1] http://www.unicode.org/Public/5.2.0/charts/CodeCharts-MulticolHan.pdf

[ahavatar (kbn00)]

Pan, Shi Zhu,

I am not a font expert, but my thought is that if any part of a font package is uglier than the system default one, it should not be installed by default and should not have a higher priority than the system default one.

[Pan, SZ (pan.sz)]

On Tue, Nov 10, 2009 at 11:38 AM, ahavatar <email address hidden> wrote:
> Pan, Shi Zhu,
>
> I am not a font expert, but my thought is that if any part of a font
> package is uglier than the system default one, it should not be
> installed by default and should not have a higher priority than the
> system default one.
>

The Chinese part of the korean font package is uglier than the system
default one. While the korean font is always installed by default,
those chinese characters are displayed as korean font in en_us.utf8.
Chinese users have suffered from that since ubuntu 4.x

The best solution may be designing a unique font for all CJK
characters, but I'm not sure if it is technically possible. In fact,
wenquanyi wants to accomplish that, and every single characters of
wenquanyi are open source and welcome for wiki-like improvement in its
web site.

The next-best solution may be that each user learn to use
fontconfig-voodoo or to edit /etc/fonts/conf.d manually. Currently it
may be the most possible solution.

[Qianqian Fang (fangq)]

Shi Zhu:

I want to clarify that the issue you report here does not conflict
with the solution to this bug.

At the very beginning, I was also afraid of the CJK Han-variant problems as
you did. Fortunately, what ahavatar is not about that. It is
about Hangul, which Zen Hei has the full coverage inherited
from Ubuntu's default Korean fonts. So, in that sense, it does not
conflict each other to make both Chinese Han characters and Hanguls
look good under en_US locale. The solution is simply like
what I outlined in my previous post. I also filed another bug
to fontconfig to mitigate the needs for setting antialias=false
when using embeddedbitmaps, see
https://bugs.freedesktop.org/show_bug.cgi?id=24960

<side-note>
As you and Arne pointed out, Han glyph shape variant (or Z-variant)
is more challenging issue. My suggested solution is to keep zh_CN
as the default shape variant (because of its huge coverage) for all
non-CJK locales, and use CJK specific fontconfig files to enable their
own respective settings under the corresponding locales. The
fontconfig-voodoo
tool is a Ubuntu implementations to link a set of files per locale.
I also proposed a solution which can make CJK files concurrently used,
you can find a lot more discussions at

http://bugs.freedesktop.org/show_bug.cgi?id=20911
and
https://bugzilla.redhat.com/show_bug.cgi?id=499902

I wish people who cares about this problem to give a serious
review and push the upstreams to adopt this scheme.
</side-note>

Pan, Shi Zhu wrote:
> For a long time when wqy was not present in ubuntu, Chinese text does
> not look good in en_US.utf8. Because they are shown as Korean or Japan
> font.
>
> So, if this change is reversed, should Chinese users report the bugfix
> itself as a bug?
>
> The point is: when locale is en_US.utf8, only *one* of CJK fonts will
> look good.
>
> If you put a Korean fonts as the highest priority, then Chinese and
> Japan text will not look good at en_US.utf8.
>
> If we put a Chinese fonts as the highest priority, then Korean and
> Japan text will not look good at en_US.utf8.
>
> If we put a Japan fonts as the highest priority, then Korean and
> Chinese text will not look good at en_US.utf8.
>
> The point is you should set the locale in order to raise your font to
> the highest priority. Otherwise, in en_US.utf8 no one could decide
> which one in CJK should be the highest priority.
>
> Or there may be another solution: make a unique font which looks good
> in all CJK characters...
>
> On Fri, Nov 6, 2009 at 12:50 PM, ahavatar <email address hidden> wrote:
>
>> Even my local is en_US.UTF8, I should be able to visit some Korean
>> websites, right? Without ttf-wqy-* packages, I have no problem in doing
>> so with the Ubuntu 9.10 system default (i.e. I haven't changed any font
>> nor locale setting except adding Korean language, but the default is
>> still en_US.UTF8)
>>
>> But with the ttf-wqy-* fonts installed, Korean fonts become broken and
>> almost unreadable. In fact, many Korean Ubuntu 9.10 users have the same
>> problem and have reported this in the Korean Ubuntu User forum
>> (www.ubuntu.or.kr) as well.
>>
>>
>
>

[ZhengPeng Hou (zhengpeng-hou)]

To Qianqian:
I'm about to disable 66-wqy-zenhei-sharp.conf in Lucid by default.
what do you think about it?

On Tue, Nov 10, 2009 at 12:57 PM, Qianqian Fang <email address hidden> wrote:
> ** Bug watch added: freedesktop.org Bugzilla #24960
>   https://bugs.freedesktop.org/show_bug.cgi?id=24960
>
> ** Bug watch added: freedesktop.org Bugzilla #20911
>   https://bugs.freedesktop.org/show_bug.cgi?id=20911
>
> ** Bug watch added: Red Hat Bugzilla #499902
>   https://bugzilla.redhat.com/show_bug.cgi?id=499902
>
> --
> ttf-wqy-microhei ttf-wqy-zenhei break Korean fonts
> https://bugs.launchpad.net/bugs/475240
> You received this bug notification because you are a bug assignee.
>

[Qianqian Fang (fangq)]

ZhengPeng Hou wrote:
> To Qianqian:
> I'm about to disable 66-wqy-zenhei-sharp.conf in Lucid by default.
> what do you think about it?
>

sounds fine to me, but what about Karmic? will you back-port it?

I will also polish the wqy font-setting GUI so that it can
make it easier to switch between bitmaps and vector rendering.

[ZhengPeng Hou (zhengpeng-hou)]

On Tue, Nov 10, 2009 at 1:28 PM, Qianqian Fang <email address hidden> wrote:
> ZhengPeng Hou wrote:
>> To Qianqian:
>> I'm about to disable 66-wqy-zenhei-sharp.conf in Lucid by default.
>> what do you think about it?
>>
>
> sounds fine to me, but what about Karmic? will you back-port it?
maybe yes, maybe no :)
>
> I will also polish the wqy font-setting GUI so that it can
> make it easier to switch between bitmaps and vector rendering.
>
> --
> ttf-wqy-microhei ttf-wqy-zenhei break Korean fonts
> https://bugs.launchpad.net/bugs/475240
> You received this bug notification because you are a bug assignee.
>

[Pan, SZ (pan.sz)]

On Tue, Nov 10, 2009 at 1:28 PM, Qianqian Fang <email address hidden> wrote:
> ZhengPeng Hou wrote:
>> To Qianqian:
>> I'm about to disable 66-wqy-zenhei-sharp.conf in Lucid by default.
>> what do you think about it?
>>
>
> sounds fine to me, but what about Karmic? will you back-port it?
>
> I will also polish the wqy font-setting GUI so that it can
> make it easier to switch between bitmaps and vector rendering.
>

Perhaps a more generic approach is desirable, the reason many people
dislike the default bitmap in wqy is: wqy bitmap song is "serif" style
font while wqy zenhei is "sans serif" style font.

Personally, I enable bitmap song only for "serif" style font and use a
SongTi style font for "serif" font family, while use wqy zenhei only
for "sans serif" style font, and another font for my "mono" style.

It should be much better to be able to choose Chinese font and to
enable/disable bitmap *separately* for sans, serif and mono family.

[Qianqian Fang (fangq)]

@Shi Zhu:

The nightly-built version (0.9.43) of Zen Hei now packs with 3 fonts: Zen Hei (pure vector with proportional Latin), Zen Hei Mono (vector only with monospaced Latins) and Zen Hei Sharp (zenhei with embedded bitmaps). I also wrote a simple GUI to set enable/disable bitmap in front of vector font, see http://wenq.org/forum/viewtopic.php?f=6&t=902

The NB can be downloaded from http://wenq.org/daily/zenhei/ , I would be glad to hear any feedback on this.

[Pan, SZ (pan.sz)]

On Tue, Nov 10, 2009 at 2:09 PM, Qianqian Fang <email address hidden> wrote:
> The NB can be downloaded from http://wenq.org/daily/zenhei/ , I would be
> glad to hear any feedback on this.
>
It seems to work, I have two concerns:

1. zenity depends on gtk+, atk, cairo, and lots of stuffs, it may at
least cause inconvenience to kde users. CLI program might be a better
choice.

2. may be better to provide choice separately for sans-serif, serif
and mono fonts.

[cablop (cablop)]

Maybe this link could help?

http://www.unifont.org/fontguide/

Adi Roiban wrote:
> ** Changed in: ubuntu-translations
> Status: New => Confirmed
>
>

[Qianqian Fang (fangq)]

Pan, Shi Zhu wrote:
> It seems to work, I have two concerns:
>
> 1. zenity depends on gtk+, atk, cairo, and lots of stuffs, it may at
> least cause inconvenience to kde users. CLI program might be a better
> choice.
>

zenhei already has a CLI setting tool: zenheiset
but it was packed as
/usr/share/doc/ttf-wqy-zenhei/examples/zenheiset
in 0.8.38-1ubuntu1

I guess when we update zenhei next time, I will ask Zhengpeng
to move zenheiset to /usr/bin/zenheiset

> 2. may be better to provide choice separately for sans-serif, serif
> and mono fonts.
>
this is possible with pre-made fontconfig files. Contribution is
also welcome.

[iGEL (igel)]

I can confirm this bug and think the solution should be backported. Hangeul doesn't just look bad, it's unreadable in a Ubuntu default installation (See attached image, left jaunty, right karmic). If you increase the font size to 13pt or more, the font looks great, but not at the standard application font size. I'm sorry for the Chinese users for having to endured foreign characters, but solving that problem by making another script unreadable is a very bad idea. I hope, the solutions in this topic solve it for all sides.

I'm quiet disappointed by Karmic. In the past, I thought that only mature features were introduced into a final release as a system default, but Karmic didn't just introduced this font, but also the very buggy ibus (at least for Hangeul, reported several bugs & switched back to scim).

[iGEL (igel)]

How about striping the broken Hangeul characters from that font and leave the rest as it is? (I have no experience with fonts, but it sounds easier than creating a new font) I think, most ppl could accept Hanja rendered with a Chinese font in en_US (or de_DE in my case), as long as they are readable. But having Hangeul in this state in the default installation is very unsatisfying.

[Zeniff (zeniffmartineau)]

I very much agree with iGEL in many ways. Korean fonts and input method were far better in Jaunty than in Karmic.

I've had to do a lot of guessing just to get readable Korean fonts by adjusting the default fonts in System>Preferences>Appearance and had to adjust separately in Firefox's Preferences>Content>Fonts & Colors>Advanced>Fonts for: Korean.

And to use reasonable fonts in NateOn (Korea's very popular messenger), using the qtconfig program in the repos (which says it's specifically for fixing CJK fonts for KDE programs in Gnome) does nothing except make a config file unused by programs... so I had to copy the relevant font parts from that to the ~/.kde/share/configkdeglobals file.

Also, IBus for Korean doesn't even have the ability to enter very commonly-used special characters like Scim and MS' IME do, yet it automatically replaces Scim...

I think it's very important for every aspect of the languages on Linux to be easily found and configurable. It's important that multiple fonts can be used at the same time (for translators, language-learners, etc). Also, it's very important for font, language, and input method options to be very easy to find and change (the Language Support program was almost the only reason I ever considered Ubuntu when I started using Linux because it was impossibly difficult to change the language when I was a new user and trying LiveCD distros).

I hope I didn't go too far off topic, sorry, but this simple issue has caused a lot of problems for me since starting Linux. Thanks for the attention to the issue~

[Scott Severance (scott.severance)]

I'd like to second iEGL's suggestion to remove Hangeul from that font. After all, Hangeul is pretty much irrelevant to Chinese, which is the focus of that font. And there is already adequate Hangeul coverage included by default without that font. Let that font do its job of handling Chinese, and let Korean fonts handle Korean (by the way, Korean rarely uses Hanja; Hangeul is used almost all of the time). Why should a Chinese font include Hangeul?

[Qianqian Fang (fangq)]

On 06/08/2010 10:58 AM, Scott Severance wrote:
> I'd like to second iEGL's suggestion to remove Hangeul from that font.
> After all, Hangeul is pretty much irrelevant to Chinese, which is the
> focus of that font. And there is already adequate Hangeul coverage
> included by default without that font. Let that font do its job of
> handling Chinese, and let Korean fonts handle Korean (by the way, Korean
> rarely uses Hanja; Hangeul is used almost all of the time). Why should a
> Chinese font include Hangeul?
>

I kindly disagree. First of all, the issue itself is only related
to an anti-aliasing setting in a fontconfig file. It has
nothing to do with the Hangul glyphs themselves
included in these font. The right solution has been
already provided in comment #8.

Secondly, because Korean people rarely use Han characters
does not mean Chinese won't use Hangul. For the same
reason, LGC people almost never use Chinese, that
won't give you a conclusion that all LGC characters
should be removed from Chinese fonts.
(Hangul are found in many Chinese ASCII arts,
and I believe many Korean literature and archives
have Hanja)

Thirdly, removing Hangul from these two fonts
gain little saving in terms of space. All Hangeul glyphs
are composed of references, and removing them
only save a few hundred KB. The price to pay is
that Chinese users have to install Korean fonts to read
Korean. That is almost tons of MB in space. A
single compact Unicode font is very important for
many mini-distributions and i18n support for open-source
games.

Lastly, Hanguls in Korean fonts and Chinese fonts
have different metrics and slightly different styles.
It is always good to have matched metrics when
displaying a text mixed with different languages.

[Qianqian Fang (fangq)]

and also, the right thing to do is to push back-porting zenhei 0.9.45 to Karmic, please see Bug#538952
https://bugs.launchpad.net/ubuntu/+source/ttf-wqy-zenhei/+bug/538952

[Scott Severance (scott.severance)]

Quick followup to Qianqian: ttf-wqy-zenhei also needs to be backported to Lucid; I never experienced this bug until I upgraded to Lucid. The version currently in the Lucid repo is 0.8.38-1ubuntu1.

[David Planella (dpm)]

There has been quite a lot of good discussion in this bug.

Could someone perhaps summarize if there is anything to be done?

Someone is mentioning that the latest ttf-wqy-zenhei package needs to be backported to Karmic and Lucid. If someone would like to contribute on that, the best thing would be to file an SRU request as explained on https://bugs.launchpad.net/ubuntu/+source/ttf-wqy-zenhei/+bug/538952/comments/5

Thanks!

[Hendrik Schrieber (hennekn)]

According to the comments this is fixed in the latest packages. Karmic and Lucid reached EOL so I am closing this as Fix Released.