Support all languages in fontconfig, language-selector is not enough

[David Nemeskey]

Binary package hint: language-selector

CJK language support in Ubuntu is substandard. Removing the fonts from fontconfig, and placing them in the language-selector package is good for modularity; however, it wasn't done right.

The first problem is that for example the ja_JP file does not include the default latin fonts, so any latin text will be ugly. I haven't checked, but I have the feeling that "exotic" letters like é, á, ő, ű (accented or double-accented e, a, o, u) would not even be displayed correctly. So I had to create my own en_JP file.

The second is, why do I have to CHOOSE between Japanese and Chinese? If I choose ja_JP, Chinese characters will not be displayed. And it is true in the other way around, too. It's very stupid.

What I suggest is that all font configurations under language-selector list the same (all?) fonts. The difference could be the order of the fonts. In a Japanese setting, the Japanese fonts would get higher priority, while in a Chinese one, the Chinese fonts. In this way, a Japanese system could display Chinese text too, and vice versa, and the different symbols with the same unicode would be displayed according to the "preferred" language.

Please have a look at the attachment: Fedora includes all these fonts in the configuration file by default, so the user is able to view Japanese AND Chinese text without changing anything.

[Arne Goetje (arnegoetje)]

At least for Hardy this should not be a problem any more. The locale specific font preferences have a "prepend" binding. That means they will be put at the beginning of the font list, which is composed by fontconfig. Other matching fonts for the same family will be appended. A list containing all available fonts on the system is therefor not necessary.
For the accented Latin characters: obviously Japanese users don't use any Latin characters besides ASCII, or their fonts would include them. We got the preferred fonts list from the Japanese community and it seems they prefer the Latin characters to be displayed with the same font as the rest of their Japanese characters.
The question why you need to choose between Chinese and Japanese is easily answered: Any CJK font in the system only contains a subset of the encoded CJK ideographs, namely those which are needed for the specific language the font was made for. An exception to this is currently only "WenQuanYi ZenHei", which contains all basic CJK ideographs which are used in all CJK regions, but uses the simplified Chinese glyph shapes. This might not suit Japanese users.
However, choosing ja_JP should give you a Japanese font for the Japanese characters plus ASCII, and other fonts on the system will be used to render the rest of the characters, including Chinese.
If you prefer a different setting than this, I'm afraid there is currently no other possibility than to hack your own fontconfig configuration together and put it into ~/.fonts.conf .

So, I'd like to set this bug to "invalid", because the functionality is there and it works in Hardy. If you still think this bug is valid, please support us with more information and a few screenshots, so that we can see how the characters are rendered on your system. Please state at least which release version and which flavor (Ubuntu, Kubuntu, Kubuntu KDE4, etc.) you are using.

[David Nemeskey (nemeskeyd)]

I do not agree with setting the font to invalid. It could be invalid, if by default I would be able to read any language. However, I am currently not able to do so.

I am using Kubuntu 7.10. When I installed my system, I could not even read Japanese text, only the characters that are in the Dejavu set (kanas and a subset of the kanjis). So I HAD to set the language via the language-selector to set it to ja_JP.

About accented characters: I am Hungarian, but I also want to be able to read Japanese pages. Why is it so strange? So I DO care about accented characters. As written in my first comment, I had to create an en_JP (ok, it could have been called hu_JP) setting, which includes the default deja/bitstream fonts for latin text.

Therefore your answer that "Japanese users don't use any latin characters besides ASCII" does not hold for three reasons:
1. There may be Japanese users, who want to use accented characters, because they are learning a European language
2. There are Western users who are learning Japanese, and who want to be able to read Japanese AND their mothertongue, too.
3. What about Chinese people learning Japanese, or Japanese people learning Chinese? Should they switch back and forth between ja_JP and zh_XX every few seconds?

With the current system settings, these users are in trouble. I do not know about Hardy; if the selection algorithm is different there, it's ok, but Gutsy is still supported, and will be for at least 1 more year, right?

But even with my (en_JP) settings, when my friend sent me Chinese text in Kopete, I could not read it. Only the characters that are common with Japanese were shown; the others became squares. Now please read the original report carefully: I understand why you have to choose between Japanese and Chinese. I just don't agree with the current state, where you have to choose between being able to read Japanese and not being able to read Chinese, or vice versa.

I think the user should be able to choose which language he/she prefers; but that should only make a difference in the order of the font list. So for example, both Kochi Mincho and WenQuanYi ZenHei would be in the list, but in the ja_JP file, Kochi would be higher in the list, and in the zh_CN settings, WenQuanYi would be higher. Since there is no way to get around the fact that there are characters that are different in Chinese and Japanese, but have the same unicode value, this is the best we can get.

Please have a look at the screenshot: Firefox is able to display the Chinese text. But it is pasted to Kopete, some of the characters (those not in Kochi) become squares. If a Chinese font would be in the list, it would be displayed in the same way as in Firefox (since Japanese fonts have priority, the kanjis found in Kochi would be displayed by Kochi -- it is not perfect, but there is no better way).

Also, I believe when you "have hack your own fontconfig", it is a proof that this bug is valid.

[Arne Goetje (arnegoetje)]

The bug was actually in qt3 not being able to use fontconfig correctly. This has been fixed in Hardy, I think. And it's not an issue anymore for qt4. Therefor marking as Fix Released.