Incorrectly rendered Telugu, Kannada, and Hindi text, even with MOZ_ENABLE_PANGO=1

[Suraj N. Kurapati]

= Problem =

In Dapper, firefox (version 1.5.dfsg+1.5.0.3-0ubuntu3) is not rendering Telugu, Kannada, and Hindi text properly, even with the MOZ_ENABLE_PANGO=1 variable set. The text appears as if MOZ_ENABLE_PANGO=1 variable was being ignored.

In contrast, gedit and mozilla-thunderbird are correctly rendering Telugu, Kannada, and Hindi text, with and without the MOZ_ENABLE_PANGO=1 variable (see attachments).

= Steps =

0. Use the Ubuntu Dapper live CD or start with a fresh installation of Ubuntu Dapper. (Note: if you use the live CD, Hindi text will appear fine, because the live CD enables the lanugage-pack-hi package. When you do an installation of a non-Hindi locale using the live CD, it prunes the lanugage-pack-hi package, and thus Hindi text will appear incorrect after booting the newly installed system.)

1. Install the ttf-indic-fonts package:

$ sudo apt-get install ttf-indic-fonts

2. Fix the general rendering of Telugu and Kannada text in Dapper (see bug 48280):

$ sudo sed -i.orig 's/lang=:/lang=te:/g' /usr/share/fonts/truetype/ttf-telugu-fonts/fonts.cache-1

$ sudo sed -i.orig 's/lang=:/lang=kn:/g' /usr/share/fonts/truetype/ttf-kannada-fonts/fonts.cache-1

3. Load some Telugu, Kannada, and Hindi text in firefox:

$ MOZ_ENABLE_PANGO=1 firefox te.wikipedia.org

$ MOZ_ENABLE_PANGO=1 firefox kn.wikipedia.org

$ MOZ_ENABLE_PANGO=1 firefox hi.wikipedia.org

4. Observe the incorrectly rendered text (see attachments).

[Suraj N. Kurapati (sunaku)]

gedit and mozilla-thunderbird are correctly rendering the Telugu text (circled in red in the firefox window) that firefox is rendering incorrectly.

[Sunil Mohan Adapa (sunil)]

Please note that the problem is because ttf-freefont which contains some incomplete Telugu glyphs is being given priority over the Pothana2000 (from ttf-telugu-fonts). Since ttf-freefont does not itself specify lang=te perhaps this solution will work. When freefont also has lang=te the problem might reappear. Moreover, the problem does not appear with out ttf-freefont even with "lang=". Qt4 (earlier betas) and Java had a problem because of the lack of lang but Pango does fine. On my system I specify that Pothana2000 should be the default font by saying having /etc/fonts/local.conf similar to the following:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
 <alias>
  <family>sans-serif</family>
  <accept><family>Pothana2000</family></accept>
 </alias>
 <alias>
  <family>serif</family>
  <accept><family>Pothana2000</family></accept>
 </alias>
 <alias>
  <family>monospace</family>
  <accept><family>Pothana2000</family></accept>
 </alias>
</fontconfig>

PS: this has nothing to do with Mozilla pango rendering.

[Suraj N. Kurapati (sunaku)]

Before the fontconfig adjustment, which gives priority to Pothana2000 font over ttf-freefont, was applied I took this screenshot for evidence. All three programs (gedit, thunderbird, firefox) were run with and without the MOZ_ENABLE_PANGO=1 variable and they produced the same result shown in this image.

Here gedit and thunderbird correctly render text (circled in red inside firefox windows) from three languages (Hindi, Kannada, then Telugu). But firefox only renders the Hindi text correctly.

[Suraj N. Kurapati (sunaku)]

After the fontconfig adjustment, which gives priority to Pothana2000 font over ttf-freefont, was applied I took this screenshot for evidence. All three programs (gedit, thunderbird, firefox) were run with and without the MOZ_ENABLE_PANGO=1 variable and they produced the same result shown in this image.

Here gedit and thunderbird correctly render text (circled in red inside firefox windows) from three languages (Hindi, Kannada, then Telugu). But firefox only renders the Hindi text correctly.

Also, there is some improvement in firefox rendering of Telugu text after applying the fontconfig adjustment. However, the result is not 100% correct. Instead, it appears like the Pothana2000 font is being used and Pango is *not* being used.

[Suraj N. Kurapati (sunaku)]

After the fontconfig adjustment, which gives priority to Pothana2000 font over ttf-freefont, was applied I took this screenshot for evidence. All three programs (gedit, thunderbird, firefox) were run with and without the MOZ_ENABLE_PANGO=1 variable and they produced the same result shown in this image.

Here gedit and thunderbird correctly render text (circled in red inside firefox windows) from three languages (Hindi, Kannada, then Telugu). But firefox only renders the Hindi text correctly.

Also, there is some improvement in firefox rendering of Telugu text after applying the fontconfig adjustment. However, the result is not 100% correct. Instead, it appears like the Pothana2000 font is being used and Pango is *not* being used.

[Suraj N. Kurapati (sunaku)]

Please see the attachments and descriptions regarding the fontconfig priority adjustment. As you can see, regardless of the fontconfig adjustment and the MOZ_ENABLE_PANGO=1 variable, I do not get correct rendering of Telugu text in firefox, but gedit & thunderbird are able to render all three languages correctly.

One interesting thing is that Hindi text is always rendered correctly in firefox. I think there is something wrong with the configuration of ttf-{telugu,kannada}-fonts packages with respect to firefox, or vice versa.

Also, for the screenshots, I made a brand new profile for Firefox to ensure that my settings don't influence the result. And sorry for posting the last attachment twice; my internet connection broke while uploading so I re-uploaded it.

[Suraj N. Kurapati (sunaku)]

Correction: In the attachements regarding fontconfig adjustment, the rendering of Hindi text in firefox is inconsistent with that of gedit & thunderbird.

In particular, you can notice the inconsistency for the vowel marker the beginning of the sentence: in gedit & thunderbird, it appears before the first consonant; whereas in firefox it appears after the first consonant. This inconsistency can also be found in the remainder of the Hindi text.

I thought the Hindi text was rendered correctly, but I cannot read Hindi. Just now, my mother confirmed that the first word reads is shown as "wikipedia" in gedit & thunderbird, whereas firefox shows it as "wakipedia" --- which is incorrect.

[Sunil Mohan Adapa (sunil)]

The following is the code from firefox-runner in drapper's firefox deb diff.

+if [ "x${MOZ_DISABLE_PANGO}" = x ]; then
+ if egrep '^(bn|gu|hi|kn|ml|mr|ne|pa|ta|te)_' \
+ /var/lib/locales/supported.d/*[^~] >/dev/null 2>&1; then
+ MOZ_DISABLE_PANGO=0
+ else
+ MOZ_DISABLE_PANGO=1
+ fi
+ export MOZ_DISABLE_PANGO
+fi
+if [ "x${MOZ_DISABLE_PANGO}" = x0 ]; then
+ unset MOZ_DISABLE_PANGO
+fi

Can you please try installing the Telugu language support and also setting MOZ_DISABLE_PANGO=0 instead of MOZ_ENABLE_PANGO=1.

We have tested Flight 5 for Drapper Drake and after Telugu langauge support is installed, firefox rendering was fine.

[Suraj N. Kurapati (sunaku)]

Thanks for your help Sunil. As you instructed, I installed the Telugu language pack and got correct rendering. Likewise, installing the Kannada language pack fixes the rendering of Kannada text in firefox.

In conclusion, installing the language packs fixes Firefox so that it renders Indian text just like Gedit and Thunderbird.

[Suraj N. Kurapati (sunaku)]

I forgot to mention, I installed the language packs like this:

$ sudo apt-get install language-pack-te language-pack-kn

Then I ran firefox without any MOZ_ variables:

$ firefox te.wikipedia.org kn.wikipedia.org

[Suraj N. Kurapati (sunaku)]

FYI, this bug can be closed now.

The previous post describes the solution: install the language-pack-XX package to get proper rendering for language XX in Firefox.

[Caroline Ford (secretlondon)]

apparently fixed