Improvement for zh-tw fontconfig settings of language-selector

Bug #713950 reported by Cheng-Chia Tseng
42
This bug affects 3 people
Affects Status Importance Assigned to Milestone
language-selector (Ubuntu)
Fix Released
High
Gunnar Hjalmarsson
Declined for Trusty by Gunnar Hjalmarsson

Bug Description

Binary package hint: language-selector

Now the setting of 69-language-selector-zh-tw.conf picks up the Latin fonts first, then fallback to Chinese fonts. By default, ubuntu only installs DejaVu font family and WenQuanYi Micro Hei font.

DejaVu is a high quality Latin fontfamily which has ExtraLight, Condensed, Book, Condensed Italic...and many more styles. WenQuanYi Micro Hei font covers Latin, Chinese, Japanese and Korean, but only has Regular, Italic, Bold and Bold Italic styles.

However, there is no need to pick up the Latin fonts first. I am here to suggest a better configuration by the reasons below:

1. As far as I have known, WenQuanYi Micro Hei font provied all the typefaces which DejaVu fontfamily covers, but lack of Condensed, EtralLight and some other styles.

2. The order of languages most used by Chinese (Taiwan) users is: Chinese > English > Japanese > Korean > other Western languages(eg. French, Spanish, German, Russia...). Ideally, the system should select the best fonts having Chinese (Taiwan) characters first, then go to the best Latin fonts to meet Chinese (Taiwan) users' need.

3. There is no evidence indicating mixing Latin characters provided by Latin fonts with Chinese characters provided by CJK fonts will be the best default experience. Actullay, that will sacrifice the consistency when there is Latin and CJK text displaying on the screen at the same time, because Latin characters and CJK characters are in different flavors. However, many users insist on using DejaVu fontfamily to display the Latin characters. The same visual appearance is reserved by the proposed configuration.

3. Selecting fonts including Chinese (Taiwan) characters first in the configuration provides the best compatibilty for Chinese characters with those programs which do not follow the fallback machenism of fontconfig, such as Evince, to be readable. The programs like that only pick up the first font, DejaVu font, and displaying squares for Chinese characters. With this improved configuration, the problem still can be found out by the "Latin characters" part which should be DejaVu but not WenQuanYi Micro Hei.

4. People still have the freedom to edit the configuration or add new rules, there is no harm selecting those font strings including Chinese (Taiwan) characters first in zh_TW configuration.

Please consider to choose the CJK fonts before the Latin fonts.

Tags: patch
Revision history for this message
poloshiao (poloshiao) wrote :

This post was original written by xenomorph0525 and was posted here:
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=163566#forumpost163566
I help translate it into English and post it to this bug report.
.................................................................................

1. about 69-language-selector-zh-*.conf
The traditional chinese users take more care about normal displaying with chinese fonts than about nice appearance or not with English fonts. Furthermore there are many subjective viewpoints that are highly diversified about being good or ugly looks related to those fonts. Some agree that it will looks better if showing chinese characters with chinese fonts while showing English characters with DejaVu fonts. Others think that it should be better if showing English characters and Chinese characters with the same fonts as well.

2. It will prevent chinese characters from being normal displayed and partially declined to be replaced with square characters in some programs if allowing strings including Bitstream Vera、DejaVu retained in 69-language-selector-zh-*.conf. We have implemented a case study and posted the report in details:
https://bugs.launchpad.net/ubuntu-translations/+bug/659280/comments/13

3. We are holding an affixing theme
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654
for survey of being favorable to or against the proposal to edit 69-language-selector-zh-tw.conf, 69-language-selector-zh-hk.conf, 69-language-selector-zh-mo.conf among the traditional chinese users. We notify adequately whether they are in favor of removal strings including Serif、Sans、Monospace of DejaVu fonts and replacement with those from chinese fonts when display English characters in traditional chinese locale.
We will gather statistics about agreement or opposition separately at 2/24/2011, one week before the release of Ubuntu 11.04 Alpha 3, and 3/24/2011, one week before the release of Ubuntu 11.04 Beta. We will post the result data here as a reference for language-selector authorities.

4. At the same time we support to edit those programs which have problems in displaying chinese characters without a doubt.

5. Our ultimate anticipation is that there should be not any mojibake by default font setting in ubuntu 11.04 traditional chinese locale. We expect to implement this wish by removing strings including Bitstream Vera、DejaVu from 69-language-selector-zh-tw.conf, 69-language-selector-zh-hk.conf, 69-language-selector-zh-mo.conf. Your kind attention will be appreciated very much.

description: updated
description: updated
Revision history for this message
hemiscy (scy-hemi) wrote :

DejaVu Serif or other Latin fonts should be kept in Serif, because the AR PL family fonts are monospaced.

Revision history for this message
Yao Wei (medicalwei) wrote :

For the reason 3, the English glyph provided in Chinese font are lack of italic typefaces (not oblique), bold fonts (sometimes, and not the fake one), and are usually ugly. It is more obvious if you read documents written in English in Chinese environment.

Revision history for this message
Aron Xu (happyaron) wrote :

For reason 1, obviously we don't like to have one font contains all typefaces for all languages, so coverage is an important thing but not the reason for "it is suitable for being default".

For reason 2, this is a quick-and-dirty workaround which causes regression as discussed in Bug #659280, so working around that problem should not be a reason in this proposal.

For reason 3, #3 has said a good excuse.

For reason 4, if you say people still have the freedom to change back, then you have the freedom to change it to whatever you like now.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

In my opinion, Traditional Chinese environment should be prepared for majority of the tradtional Chinese users. If what we do can provide a better default enviroment for the most, we should just do it.

I would like to know the orginal idea behind the lanugage-selector configuration from the designer or developer.

Revision history for this message
poloshiao (poloshiao) wrote :

This post was originally written by xenomorph0525 and was posted here:
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=164272#forumpost164272
I help translate it into English and post it to this bug report.

................................................................
The file 69-language-selector-*.conf first appeared since Ubuntu 8.04 was issued. There were not any problem like Sans fonts becoming sqare characters when displayed chinese fonts before that time. It is owing to the way of mixing Chinese and English fonts in a file 69-language-selector-zh-*.conf that the sqare fonts came into being.

It is caused by 2 main facters that Sans fonts becoming sqare characters when displayed chinese fonts:
1. the way of mixing Chinese and English fonts in the file 69-language-selector-zh-*.conf
2. not in support of fallback function like the file 69-language-selector-zh-*.conf by some applications.

The problem should be resolved thoroughly only if removal of these two factors. In any case we have to remove the strings including Bitstream Vera、DejaVu from the file 69-language-selector-zh-*.conf so that other applications will not show abnormal Chinese fonts caused by the second factor.

After all it is so hard to control in advance how the programmers design their software. At least the first factor is under our master control among these two factors. Of course we should edit those programs with known bugs in the 2nd factor, beside the first factor, either.

It must not be one kind of good setting to mix Chinese fonts and English fonts in one setting file.
This would be regarded, by the users, as to be drawn back to normal setting ( not mixed Chinese and English fonts in one file) in case that the proposal to edit the file 69-language-selector-zh-*.conf was brought up when Ubnutu 8.04 had been releasing.

Maybe the reason why it is classified into not good solution by somebodies should be too late, after 2 years, to be brought forward to edit the setting. Nevertheless it is because of time point to shape the different impression. In fact it is still a suggestion of such kind of item as regressing back to normal setting.

Revision history for this message
Aron Xu (happyaron) wrote :

Discussions may continue only about why your proposed appearance on how the fonts are displayed would be better than the current one and fit for the majority of people.

Any reasons about "it will avoid applications which don't support fallback" should just stop: go and fix the buggy applications. For example if you like to fix a related PDF rendering issue, please go to Bug #659280 and put your efforts on pushing/fixing poppler(-data).

Revision history for this message
poloshiao (poloshiao) wrote :

There are 12 users who support favorablely to the affixing theme at
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654
, but 1 user who exptresses against the proposal to edit 69-language-selector-zh-tw.conf.

There are totally 13 users who signed their opinions about the the affixing theme to edit 69-language-selector-zh-tw.conf、69-language-selector-zh-hk.conf、69-language-selector-zh-mo.conf on http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654.

You are welcomed to visit the the affixing pages at http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654&forum=2.

The next statistics will be reported here on 03/24/2011.

Revision history for this message
Wei-Cheng Pan (legnaleurc) wrote :

This is definitely a painful problem for Chinese users, but such
suggestion is not the right way.
IMHO, first of all, this is a bug which belongs to other packages.
Also, this is a workaround, not a really robust solution.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

Well, this is a suggestion on having better zh-tw fontconfig settings, and not specific to solve the problems which programs show squares for Chinese characters.

Please just give some advice to have a better zh-tw fontconfig setting, and not to mix this bug report with other issues. If you think that the current setting is quite good now, just say that you accept the current setting.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

According to #3 Ming-Ting Yao Wei, keeping oblique, bold and italic tyles of Latin chracters might be needed by some Chinese (Taiwan) users, so I think that selecting fonts having Chinese (Taiwan) characters before Latin fonts are better than removing Latin fonts from the configuraiton.

The reason to select fonts having Chinese (Taiwan) characters first:

The order of languages most used by Chinese (Taiwan) users is: Chinese > English > Japanese > Korean > other Western languages(eg. French, Spanish, German, Russia...)

Idealy, the system should select the best fonts including Chinese (Taiwan) characters first, then go to the best Latin fonts to meet Chinese (Taiwan) users' need. They will get CJK and Latin from WenQuenYi Micro Hei first, then get the expanded font type (oblique and italic) from DejaVu.

I propose to select fonts in this order:

* serif
AR PL UMing TW > AR PL UMing HK > AR PL ShanHeiSun Uni > AR PL New Sung > WenQuanYi Bitmap Song > AR PL UKai TW > AR PL UKai HK > AR PL ZenKai Uni > DejaVu Serif > Bitstream Vera Serif

* sanss-serif
WenQuanYi Micro Hei > Droid Sans Fallback > WenQuanYi Zen Hei > AR PL UMing TW > AR PL UMing HK > AR PL ShanHeiSun Uni > AR PL New Sung > HYSong > AR PL UKai TW > AR PL UKai HK > AR PL ZenKai Uni > DejaVu Sans > Bitstream Vera Sans

* monospace
WenQuanYi Micro Hei Mono > Droid Sans Fallback > WenQuanYi Zen Hei Mono> AR PL UMing TW > AR PL UMing HK > AR PL ShanHeiSun Uni > AR PL New Sung > HYSong > AR PL UKai TW > AR PL UKai HK > AR PL ZenKai Uni > DejaVu Sans Mono > Bitstream Vera Sans Mono

description: updated
Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

The fonts having the characters of most used language should always be selected prior to the second one and so one to expand the character coverage step by step. It just makes no sense to select other fonts of minor used languages before the major ones.

If developers of Ubuntu think that CJK fonts lack of oblique, bold, book, condensed or else styles to get better support for Latin typography is quite a problem and should be fixed in this configuration, I propose set other configuration to make up for the defects of the CJK fonts after the "font selection order" part.

For example, if developers want to use "ubuntu" font as the first one being choosed by programs for sans-serif, just set a strong binding for it after the priority settings in order to seperate the "font selection order for specific language users" and "do some tricks to have better looking, more extended support for Latin typography and the preference for specific fonts" ideas.

description: updated
Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

I just checked 69-language-selector-ko-kr.conf.

The configuration has matched the rule I proposed above, so what I said in #12 is prooved to work.

Dear developers, please CONSIDER to do that for zh-tw, too!

Revision history for this message
poloshiao (poloshiao) wrote :

There are 15 users who support favorablely to the affixing theme at
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654
, but 4 user who exptresses against the proposal to edit 69-language-selector-zh-tw.conf.

There are totally 19 users who signed their opinions about the the affixing theme to edit 69-language-selector-zh-tw.conf、69-language-selector-zh-hk.conf、69-language-selector-zh-mo.conf on http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=169324#forumpost169324.

You are welcomed to visit the the affixing pages at http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654&forum=2.

The next statistics will be reported here on 04/21/2011.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2011-03-23 18:57, Cheng-Chia Tseng wrote:
> I just checked 69-language-selector-ko-kr.conf.

I made suggested changes to that file be uploaded a few weeks ago, since
there was a patch and people seemed to be agreed.
https://launchpad.net/ubuntu/+source/language-selector/0.15

> The configuration has matched the rule I proposed above, so what I said
> in #12 is prooved to work.
>
> Dear developers, please CONSIDER to do that for zh-tw, too!

Any chance that someone can make a patch, which people are reasonably
agreed on would be an improvement if applied?

Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

I am a Traditional Chinese User, and this is my personal opinion.

I also don't want to edit 69-language-selector-zh-tw.conf, and I agree with Aron Xubug , the repor and fix should goes to upstream application. the root cause is those application can not show Chinese word correctly does not support font-config.

To solve this, I need to give input to upstream, not modify the 69-language-selector-zh-tw.conf to avoid the problem.

and also, to modify 69-language-selector-zh-tw has a risk that Traditional Chinese can not find which application does not support i18n well, and never give feedback to upstream application, so upstream will not support i18n until a non-latin language developer joined.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

It's a pity that some programs do not work well with fontconfig configuration. We cannot prevent those programs from not respecting fontconfig setting. Programmers have the responsibility to design apps following standards.

We cannot give up a chance to improve the configuration while it can be done in a better way, although it makes some buggy programs seem to work correctly. This is not the fault of this proposed configuration, but is the fault of the programs.

Chinese (Traditional) users should have the right to get the improved configuration by default.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

For what it's worth, I tend to agree with Cheng-Chia Tseng. It makes more sense to me to build an OS for the real world, than for a perfect world that will never come true.

Subscribed Martin Pitt, since I think this is ultimately a policy matter.

Revision history for this message
Aron Xu (happyaron) wrote :

So the discussion are dating back to the compatibility of broken applications, but this bug is not relevant. ;-)

If you want to say "I give it a reason to change the fonts for it's better in look and feel", but your ideas behind is "it will fix the breakages and I do not care the look and feel that much", then you misunderstood the problem.

Please point out real life applications which does not work with the fontconfig settings? If there are many then we should change fontconfig now; if there are only a small number of applications affected, then just go and fix them.

To be a bit informal , I'm curious why you pay so much time arguing about changing the fonts for you cannot read PDFs (please, don't say there are others if you cannot point out many other _live_ examples), you can just fix poppler(-data) and all the people are satisfied. The time you have spent on arguing might be already enough for fixing it.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

Nope, I am not talking about the compatibility. Actually Hsin-Yi Chen (hychen) talks it in #16.

The bugs are still valid (programs showing squares) even we use a better configuration. (you can see http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654&viewmode=flat&order=ASC&type=&mode=0&start=256 in Chinese)

I was talking about "some buggy programs seem to work correctly" should not be a blocker to have a better and more reasonable configuration for most of the Chinese (Traditional) users.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

This is an improved configuration for zh_TW.

Good part:
1. The best fonts having the characters of most used language are selected prior to the second one and so one to expand the character coverage step by step. (Chinese > English > Japanese > Korean > other Western languages(eg. French, Spanish, German, Russia...)

2. It provides the same visual appearance as the original one. (Latin characters by DejaVu + CJK chracters by WenQuanYi Micro Hei.) Many users insist on that this visual appearance is better than Latin + CJK characters all by WenQuanYi Micro Hei.

3. It separates "do some tricks to have better looking, more extended support for Latin typography and the preference for specific fonts" idea from "font selection order for specific language users".

Controversial part:
Some programs not following the fontconfig settings seem to work properly when only Chinese characters are used in the content. (Most users may accept this side effect, while some commenters here may not.)

However, "Bind WenQuanYi Micro Hei with DejaVu" part in the improved configuration is still not supported by these programs, they will show the Latin parts of WenQuanYi Micro Hei but not of DejaVu. And the missing characters of WenQuanYi Micro Hei won't still be substituted by the later ones in the order list, just displaying "white space" for those characters. Those programs not supporting fontconfig settings well can still be found by these clues.

People who are interesting in testing my proposed file are welcome to test it if it provides the same user experiment for Chinese (Taiwan) user or not. Advice is always welcome, too.

Keng-Yu Lin (lexical)
Changed in language-selector (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

@Cheng-Chia Tseng

Can you provide the test case and result screen shot, then I can reproduce the result easier

here is the example of Test Case:

==========================Test Case Example Start ======================================================
Distribution: Ubuntu Maverick
Packages: ibus-1.3.7-1ubuntu3, ibus-1.3.6.20100730-1

TEST CASE:

1. ibus is running
2. select ibus-chewing
3. type the character 安, normally I should only have to type "-" on the US keyboard, which is the equivalent of ㄢ or "an" in pinyin
4. press Enter to send Chinese character has first tone

Except Result:

   output Chinese character has first tone

Actual Result:

   not Chinese character has first tone output
========================Test Case Example End=========================================================

Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

@Aron Xu

Cheng-Chia Tseng is talking about "Is it reasonable that Chinese should font be a first priority in fontconfig conf for Chinese users"

I agreed with him in this part but also said we need to think the risk of this change.

so, the question now to me is "does this change bring new side effect or bugs?",
in my opinion, we need to prepare a lot of test cases for this proposal, and then to verify does the new configuration work or not.

the target of Improvement configuration should pass this 2 test basically.

1. User still can find buggy application
2. The latin and Chinese word shows correctly and its look as the same as before

@Cheng-Chia Tseng

correct me if I misunderstand.

if not , please modify the summary because this change is not aid to fix those
application does not support font-config fallback machenism

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

Case 1: File Manager display
Result: the same

Screenshot for current setting: http://www.flickr.com/photos/pswo10680/5567004925/
Screenshot for proposed setting: http://www.flickr.com/photos/pswo10680/5567588444/

Case 2: evince with a test PDF
Result: neither has the correct output

Screenshot for current setting: http://www.flickr.com/photos/pswo10680/5567004947/
Screenshot for proposed setting: http://www.flickr.com/photos/pswo10680/5567588500/

The test PDF can be downloaded at: http://www.mediafire.com/?ac9ck56bfts59w6

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

Test Case 1:
1. use the setting you would like to test
2. open File Manager
3. take a screenshot

Test Case 2:
1. get the test.pdf
2. use the setting you would like to test
3. open test.pdf in evince
4. take a screenshot

Revision history for this message
Keng-Yu Lin (lexical) wrote :

@Cheng-Chia Tseng,

  Can you try install the package "poppler-data" and try to open the test pdf with evince again?

description: updated
Revision history for this message
Keng-Yu Lin (lexical) wrote :

For a more complete review and discussion, is there a list of the software not-working well (besides evince)?

It is good to collect the information so we can see if this is common issue among those softwares or it is caused with different sources.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

I have installed poppler-data before the test case.

Revision history for this message
Martin Pitt (pitti) wrote :

@Gunnar: Unfortunately I have about zero experience with CJK font configuration, so I don't have an opinion here. I'm happy to accept any configuration changes which are agreed upon by Keng-Yü Lin, Aron Xu, and Hsin-Yi, Chen.

Revision history for this message
Martin Pitt (pitti) wrote : Re: [Bug 713950] Re: Improvement for zh-tw fontconfig settings of language-selector

> Can you try install the package "poppler-data" and try to open the
> test pdf with evince again?

For the record, language-selector in natty now installs poppler-data
for CJK.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

@Cheng-Chia Tseng

I want add third test case which the your configure MUST pass, this test is aim to check ALL software does not support i18n yet can display English correctly in zh_TW environment.

Step to Reproduce:

1. Install the software does not support fontconfig without zh_TW translation
2. set LANG=C, which means current we are not in zh_TW locale
3. launch the installed software

Except Result:

The software *MUST* shows English gylth correctly

Possible Result We Don't Want:

The software shows a square in which the gylth should be displayed

I will provide the result when I'v done the test, I need to find a application does not support fontconfig.

Revision history for this message
Keng-Yu Lin (lexical) wrote :

re-attach the test pdf

Revision history for this message
poloshiao (poloshiao) wrote :

A report digested from an earlier post related to a test with poller-data

Gunnar Hjalmarsson pointed out in
https://bugs.launchpad.net/ubuntu-translations/+bug/659280/comments/36:
poppler-data has been added to the dependency list of language-selector

Keng-Yü Lin also ask in
https://bugs.launchpad.net/ubuntu/+source/language-selector/+bug/713950/comments/26
Can you try install the package "poppler-data" and try to open the test pdf with evince again?

Thank you for your kind attention to these sqare character problems.

Both of you are supposed to be interesting in the behaviors of chinese fonts with poppler-data installed, so we digest a short report from an earlier post https://bugs.launchpad.net/ubuntu-translations/+bug/659280/comments/13 and show it here for your reference:

The original test report was finished by xenomorph0525 and posted at http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=162666#forumpost162666:
...(omitted)
2-2.with poller-data installed:
a lot of the characters appeared square, but a few characters shown normal.
http://a.imageshack.us/img195/7081/evince2281poppler.png
2-3.69-language-selector-zh-TW.conf with Bitstream Vera、DejaVu string removed (but with poller-data installed) :
all characters shown normal
http://a.imageshack.us/img196/2198/evince2281poppler69.png
...(omitted)
3-2.with poller-data installed:
http://a.imageshack.us/img573/7048/xpdf302poppler.png
3-3.with poller-data installed but 69-language-selector-zh-TW.conf with Bitstream Vera、DejaVu string removed:
http://a.imageshack.us/img3/4677/xpdf302poppler69.png
...(omitted)

Note: the original url is something like http://imgxyz.imageshack.us/img573/7048/xpdf302poppler.png
I change "imgxyz" to "a" so that you may browse them easily.

A short conclusion with poller-data:
There are still sqare characters shown in the documents or Mojibake in the path name.

Wish this disturbing problem will be solved ASAP.
Best Regards

Revision history for this message
Arne Goetje (arnegoetje) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 03/29/2011 12:19 PM, poloshiao wrote:
[...]
> A short conclusion with poller-data:
> There are still sqare characters shown in the documents or Mojibake in the path name.
>
> Wish this disturbing problem will be solved ASAP.

So, this mean, libpoppler is one of those buggy libraries, which cannot
deal with font fall backs in fontconfig.
This needs to be fixed upstream!

Can someone who can reliably reproduce this problem please file a bug
upstream for libpoppler, attach the test document and the fontconfig
configurations, and write a detailed test case what is expected and what
actually happens?

<explanation>
This bug appears because when the document was produced, the fonts which
have been used to produce the document, have not been embedded into the
document. This is bad practice in general, since in such cases the PDF
viewer needs to use system fonts to render the text in the document.
However, the font metrics may not be the same (and usually aren't) as
the ones the original creator of the document has used. As a result the
document will most likely look different than originally intended. This
is the case which happens here.

Poppler-data only provides the font metrics for CID encoded fonts.
Therefor the data will only be used if CID encoded fonts (e.g. the ones
provided by Adobe) have been used to create the pdf.

If the document was created with any other font which is not installed
on the viewer's system, libpoppler tries to use fontconfig to substitute
the font information in the document with what is available on the
system. However, it uses only the first font it finds and doesn't check
if the required glyphs are actually available. This is the bug we are
having here. If the required glyphs are not available, libpoppler should
try the next font in the fontconfig list.
</explanation>

Thanks a lot! :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJNkYE1AAoJENiwmJtstTzsI20P/R/kfpCms9Sd38MgFJAFY3k+
/PgHsBSPVQNkQvN2JOSDT6Q46LF5+uUQeLxnaSjQ+gWjsdx3eCwPB9h915B4u1E1
vqV0cknJVpIRYNJUYfb/HNG3MrjRKC8vhpOxT5Tvn1ZK6xHEmYe/REGZyhA+zWvc
VBGt22X9dHZ2aSB7GBmNyLJa1ugMjkKMvu05ohq+o4yPkd9dkZNNqPjh9pJ670Ts
MpCpWIDA2MwRlZ2ZOu+wXC7uEH1i7R3ubfZId/AXAP6UiYHOXQFkiay9QcZiHQP9
Ws8asgL8BpjMvY1+NHYUYuTxyndU9IP63Cga/IWOpdMlMHGs/ZBDnXBQlig+FCJ9
rZJiAnYDbut6D8z4xA5EvneZbs6L5ZPUgBRS8EpsKU2fUYTKbol2wcc3e0QkrZS5
GeSSS8FYPVtC782ohZndZqD9jEpzRXQs0LA0L4e5IoOdmzK5/pMywSZl6imVAXGw
Hqv+O5/XpfibxEAo5d2gf2Y63gIldLJgTF76AEr5A1ZAEb38y2iABQuRaO8IRmkB
bpLClKQwWKD4BRs0ToR9aJefyu/ws0q7e1VgtXIYvaCe9pR+AwuL2llT+9VtvrJg
JDya7pNmiGrpOM/NXjbBNdpmNkkTOUyWiAs6oar/1A0UnY2JsePTpzKbFaqTk+K1
h8QDsh09PRxgU1+oseaW
=eGeJ
-----END PGP SIGNATURE-----

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

To Hsin-Yi, Chen (hychen):

Test without Chinese (Taiwan) translation is OK. But I don't think you have to test in LANG=C, because this file should only be provided to the users who choose to use Chinese (Traditional) support in Language support. (Or maybe I am wrong?!)

Revision history for this message
poloshiao (poloshiao) wrote :

sorry for my mistyping:
#33 line 12:
Both of you are supposed to be interesting in the behaviors of chinese fonts with poppler-data installed,
must be edited to:
Both of you are supposed to be interested in the behaviors of chinese fonts with poppler-data installed,

Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

@Cheng-Chia Tseng

Sorry, I think that is my misunderstanding. you are correct.
I will test your configuration in Maverick, Natty this Sunday.

Revision history for this message
Hsin-Yi, Chen (hychen) (ossug-hychen) wrote :

@Cheng-Chia Tseng I can confirm the proposed config make the screen looks the same with current config , and applications without fontconfig support display Latin Character in WenQin Yi which is fine to me (the latin character displays correct)

I think the proposed config is reasonable/workable to me,
thanks for your effort.

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

@Hsin-Yi, Chen
Thanks for your hard work, too! :)

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

This is the diff file.

tags: added: patch
Revision history for this message
poloshiao (poloshiao) wrote :

It is 2011/04/21 now, just one week in advance of the scheduled date to release ubuntu 11.04.

The final statistic ended till 2011/04/21 about the problem to improve the square characters when chinese fonts shown is reported as follows:

There are 22 users who support favorablely to the affixing theme at http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654, but 6 user who exptresses against the proposal to edit 69-language-selector-zh-tw.conf.

There are totally 28 users who signed their opinions about the the affixing theme to edit 69-language-selector-zh-tw.conf、69-language-selector-zh-hk.conf、69-language-selector-zh-mo.conf on http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654&forum=2&post_id=175120#forumpost175120.

You are welcomed to visit the the affixing pages at http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?topic_id=35654&forum=2.

As one of the most popular linux distributions and one of the developing teams emphasizing to keep user-friendliness in mind, the abve statistic do mean something important to the ubuntu developer team.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@poloshiao
Thanks for your efforts with the poll!

On 2011-04-22 03:44, poloshiao wrote:
> It is 2011/04/21 now, just one week in advance of the scheduled date
> to release ubuntu 11.04.

Yes, but even if there had been no voices against the proposed change,
it's pretty intrusive. Uploading it right before a release would be
irresponsible, and won't happen.

There is a similar issue with the Japanese fontconfig settings
(bug 759882). I have agreed with Fumihito YOSHIDA to propose that the
equivalent Japanese file is replaced soon after the Natty release. If
approved, that way we will get a more solid base for decision through
reactions from developers and experienced users. It will be possible to
reconsider and revert the change before the release of Oneiric if doing
so proves to be motivated.

Would such a line of action with respect to traditional Chinese be an
acceptable way to handle the differences of opinion?

Revision history for this message
Cheng-Chia Tseng (zerng07) wrote :

Yes, It is kind of late to arrive with Natty.
I accept that the improved file released after Natty release.

BTW, comments are always welcome.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2011-04-22 17:59, Cheng-Chia Tseng wrote:
> Yes, It is kind of late to arrive with Natty. I accept that the
> improved file released after Natty release.

Good, then we are agreed on that.

It would be valuable to know if the trial approach I mentioned would be
acceptable also to those who are not so enthusiastic over the proposed
change.

Changed in language-selector (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
status: Confirmed → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package language-selector - 0.43

---------------
language-selector (0.43) oneiric; urgency=low

  * data/LanguageSelector.ui:
    Expand the height of the combo box on the "Language" tab.
  * fontconfig/69-language-selector-zh-tw.conf:
    Make Chinese fonts be selected before Latin fonts (LP: #713950).
    Thanks to Cheng-Chia Tseng for the patch!
 -- Gunnar Hjalmarsson <email address hidden> Wed, 13 Jul 2011 17:00:04 +0200

Changed in language-selector (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

69-language-selector-zh-tw.conf has been changed as suggested, and judging from the bug summary, which only mentions zh-tw, this bug is now fixed. However, in comments #1 and #8 also 69-language-selector-zh-hk.conf and 69-language-selector-zh-mo.conf are mentioned.

Can somebody shed some light on this please? Should the files for zh-hk and zh-mo be changed as well, and if so, exactly how? Please note that they did not exactly equal the zh-tw file before the change.

Revision history for this message
poloshiao (poloshiao) wrote :

Much regards from ubuntu-tw.org to Mr. Gunnar Hjalmarsson.
We have pasted a post on Hong Kong LoCo Team Forum
http://ubuntuforums.org/showthread.php?p=11048482#post11048482
to ask them about their opinion relative to how to edit
69-language-selector-zh-hk.conf.
We wish to have a conclusion reported to you within 10 days.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks, poloshiao, for letting us know.

If it's decided to request that the Hong Kong file is changed (and the Macao file..?), please file a separate bug about it, since this bug report was closed with the fix.

Revision history for this message
poloshiao (poloshiao) wrote :

1. The opinions to edit the file 69-language-selector-zh-hk.conf:
There is only one positive response to edit the file 69-language-selector-zh-hk.conf and no other opinion shown on HongKong Ubuntu Forums.
http://ubuntuforums.org/showpost.php?p=11049945&postcount=2
There is another post good for editing the file 69-language-selector-zh-hk.conf on http://www.ubuntu-tw.org/.
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=193370#forumpost193370
Both of them agree to edit it with the same way as to edit 69-language-selector-zh-tw.conf.

2. The opinions to edit the file 69-language-selector-zh-mo.conf:
I am sorry to say that I can not find anywhere to post the opinion survey relative to editing the file 69-language-selector-zh-mo.conf.
In my personal opinion only, I suggest to edit it with the same way as to edit 69-language-selector-zh-tw.conf
Because Traditional Chinese is mostly popular for a long time in Macao like Hong Kong and Taiwan.

Best regards from all Chinese users to Mr. Gunnar Hjalmarsson.

Revision history for this message
poloshiao (poloshiao) wrote :

correction to #49
There is another post good for editing the file 69-language-selector-zh-hk.conf on http://www.ubuntu-tw.org/.
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=193370#forumpost193370

The post link should be http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=193238#forumpost193238

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks for the info.

I see that there is a separate translation for Hong Kong Chinese, maintained by a separate translation team. This is something I wasn't aware of when posting comment #46, and now it makes me think that it's most appropriate to not change the Hong Kong fontconfig file until until such a change is requested by the Hong Kong users community directly.

As regards 69-language-selector-zh-mo.conf, I guess it may well be deleted for now, because there is neither a translation nor a locale for Macao in Ubuntu.

My apologies for having you ask around in vain. Your efforts in this matter are still much appreciated. I hope that most Taiwan users prove to like the changes to the zh-tw file.

Revision history for this message
poloshiao (poloshiao) wrote :

Agree with your final solution proposed on #51.
Thank you for your kind attention !

Revision history for this message
poloshiao (poloshiao) wrote :

I have posted your #51 on ubuntu Hong Kong Forum as follows:
http://ubuntuforums.org/showpost.php?p=11103254&postcount=3
so that they will propose their request directly if necessary.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2011-07-31 02:08, poloshiao wrote:
> I have posted your #51 on ubuntu Hong Kong Forum as follows:
> http://ubuntuforums.org/showpost.php?p=11103254&postcount=3
> so that they will propose their request directly if necessary.

Excellent, thanks!

Revision history for this message
Walter Cheuk (wwycheuk) wrote :

I would prefer keeping the Bitstream Vera and DejaVu entries, just move them to a lower priority. It neither solve the problem nor make it worse, but could leave a reference so that users can change the settings back easier; also these two fonts may provide extra characters/glyphs as they evolve.

Revision history for this message
Walter Cheuk (wwycheuk) wrote :

The proposed solution has already cater this. I am sorry for mentioning it again.

Revision history for this message
Keng-Yu Lin (lexical) wrote :

@Gunnar

the patch in comment 21 is not in Trusty. can you merge the patch for Trusty?

Changed in language-selector (Ubuntu):
status: Fix Released → Triaged
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-04-08 16:16, Keng-Yu Lin wrote:
> the patch in comment 21 is not in Trusty. can you merge the patch
> for Trusty?

Well, the simple answer is no.

This bug report was closed almost three years ago, and things have evolved since. In trusty Droid Sans is the main font for rendering Chinese, and the latest relevant bug report is bug #1227034. If there is a need for further tweaking, we have to start from the current configuration.

Please add a comment to bug #1227034, specifying what the problem is, if you have found issues with the latest Chinese font configuration.

Changed in language-selector (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.