Set $LANGUAGE if the user picks a different locale in gdm, so that language-selector and gdm stop disagreeing

[David Planella]

<SRU nomination addition>
I'm about to nominate Lucid and Maverick fixes of this bug for stable release updates (SRUs), which is the reason for this addition to the bug description.

Many (most?) new Ubuntu users, who need more than one locale to set their preferences for language and various "regional formats" (date formats etc.), experience that their selections in language-selector and/or GDM give unexpected results (see below for examples). The number of duplicates of this easily reproducable bug is not insignificant. Furthermore, IMO it's reasonable to assume that to people with no previous experience from a GNU/Linux OS it's difficult to compensate for the bug by fixing it manually. For those reasons, and even if the bug doesn't exactly represent a security issue, I think it is such a "high-impact bug" that we should not want to keep distributing with the latest LTS respective latest stable release. You can only make one first impression to those who decide to try Ubuntu. ;-)

The Natty fixes were uploaded a few weeks ago, and no worrying bugs due to the changes have been reported so far. The branches I now suggest to be uploaded to -proposed were uploaded to my PPA, and the resulting builds have been tested successfully.
https://launchpad.net/~gunnarhj/+archive/locale-test
I would say that the risk for regressions is low.

Messures have been taken with the aim to prevent undesired surprises at first login after update.
https://launchpad.net/ubuntu/+source/gdm/2.32.0-0ubuntu3

/ Gunnar Hjalmarsson

</SRU nomination addition>

Binary package hint: gdm

This is a follow-up to bug 407300, which has been fixed but a separate issue remains. I'm opening a separate task for language-selector, as it refers to the interaction between it and gdm.

The problem is basically that GDM seems to always override the LANG values set by language selector, and quite easily one can get to a situation where LANGUAGE and LANG differ and the desktop is a mixture of two languages.

Steps to reproduce (a):

 * New install, choosing Catalan in the installer
 * I log in without doing any changes to the language in GDM
 * I start System > Administration > Language support
 * I choose English there
 * I log out
 * I log back in without doing any changes in the GDM language chooser
 * My session is half English and half Catalan due to LANGUAGE=en and LANG=ca_ES.utf8 (Firefox in Catalan, gnome-panel in English, gnome-menus in Catalan).

Steps to reproduce (b):

 * Perform a full installation in English, as per http://testcases.qa.ubuntu.com/Install/NonEnglishLanguage#Installation%20Full%20Network%20Support
 * Go to System > Administration > Language Support
 * Install the Traditional Chinese language
 * Bring Traditional Chinese to the top of the list to become the main desktop language
 * Press the "Apply System-wide..." button
 * Reboot
 * When entering the session, you'll notice the desktop half translated in English, half in Chinese. The most noticeable parts shown in English are all the menus and Firefox. These applications seem to ignore the LANGUAGE variable
 * Running 'locale' on the terminal shows that LANG=en_US.UTF-8 and LANGUAGE=zh_K:en_US:en

I understand that this might be a problem in each application, as they should give LANGUAGE preference. Rather than filing a bug in each app right now, would it not make more sense to ensure that at least the first locale in LANGUAGE, the one in LANG and LC_MESSAGES are the same? (assuming it is possible to do such a thing, of course).

Also see the related bug 552664

[Kevin Huang (wasikevin)]

Tried Beta 2 on April 9th. It becomes more interesting.

reproduce steps.

1. enable simplified Chinese. reboot. The localization on menu is good.
2. enable Thai. reboot. The menu is mixed in Thai and in Simplified Chinese. Screen shot and locale are attached.

[Launchpad Janitor (janitor)]

This bug was fixed in the package language-selector - 0.5.5

---------------
language-selector (0.5.5) lucid; urgency=low

  * QtLanguageSelector.py: fix crash when no language is selected
    in the install window (LP: #553729)
  * LanguageSelector.py: if gdm is running, write the user LANG setting
    to ~/.dmrc and /var/cache/gdm/$USER/dmrc (LP: #553162)
  * updated translations from launchpad
  * remove dangling ImSwitch symlinks on startup (LP: #500594)
  * check for write permission on ~/.profile (LP: #560881)
  * check for read permission on /etc/default/locale and
    /etc/environment (LP: #554617)
 -- Arne Goetje <email address hidden> Thu, 15 Apr 2010 23:41:40 +0800

[Kevin Huang (wasikevin)]

Issue still exists in 2010-04-18 image.

Reproduce steps.

1. Download daily build from cdimage on 2010-04-18
2. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
3. reboot system after installation.
4. The language of menu is in English which should be Japanese. please see Screenshot in the attachment.

5. Connect internet and install the uninstalled language packages. Please see screenshot-1 in th attachment.
6. reboot system after installation. The language of menu is in Japanese.

6. I wan to use English UI. Therefore I move English (United States), then reboot the system. After system the rebooted, the language in menu is still in Japanese. Please see screenshot-2 with locale info.

7. In Language Support, move "English" after "English (United States)", then reboot the system. After system the rebooted, the language in menu is in English. Also, while system reboot, system pop up an window asks user if user wants to change the folder name (Documents, music......)

[Martin Pitt (pitti)]

After those steps, can you please copy&paste the output of

  echo $LANG
  echo $LANGUAGE
  cat ~/.dmrc
  cat /var/cache/gdm/$USER/dmrc

Thanks!

[Kevin Huang (wasikevin)]

reproduce with Lucid-RC

1. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
2. reboot and connect to internet. the language of menu on the top panel is in English which should be in Japanese.
Screenshot: locale and other information provided.
3. System pops up the language installation is incomplete. Download, install the language pack.
Screenshot-1:
4. Reboot the system. the language of menu on the top panel is in JAPANESE.
screenshot-2: locale and other information provided.

5. Install Traditional Chinese Language pack, move Chinese (Taiwan) to the first one in the language options
6. Reboot the system. the language of menu on the top panel is in Traditional Chinese, EXCEPT Firefox browser
 is in Japanese screenshot-3

One new bug here. originally, I saved the screenshot file on the desktop. I moved the file on the screen, it suddenly disappeared. I checked the desktop folder, the screenshot files are still there.
screenshot-4
screenshot-5: locale and other information provided.

6. move English (United States) to the first one in the language options.
Reboot the system. the language of menu on the top panel is still in Traditional Chinese, EXCEPT Firefox browser is in Japanese. Strange, all the files in desktop folder are showed again.
screenshot-6
screenshot-7: locale and other information provided.

7. reboot one more time. no change. move move English after English (United States) in the language options
screenshot-8

8. reboot system. the language of menu on the top panel is in English. however the folder name is in Traditional chinese. Usually system shoudl pop up a screen which ask users if they want to keep the old name, but no this time.

screenshot-9: locale and other information provided.

[David Planella (dpm)]

Thanks for the detailed info Kevin.

El dv 23 de 04 de 2010 a les 08:16 +0000, en/na Kevin Huang va escriure:
> reproduce with Lucid-RC
>
> 1. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
> 2. reboot and connect to internet. the language of menu on the top panel is in English which should be in Japanese.
> Screenshot: locale and other information provided.

This is expected behaviour. Due to space constraints, we can only fit a
few language packs on the CD, and Japanese is one of them.

These can be downloaded post-installation (well, or automatically during
the installation, if there is network and Internet connectivity), which
is why we show the notification message below.

> 3. System pops up the language installation is incomplete. Download, install the language pack.
> 4. Reboot the system. the language of menu on the top panel is in JAPANESE.

Yes. Up until here this is expected behaviour. See
http://testcases.qa.ubuntu.com/Install/NonEnglishLanguage#Installation%
20No%20Network

>
> 5. Install Traditional Chinese Language pack, move Chinese (Taiwan) to the first one in the language options
> 6. Reboot the system. the language of menu on the top panel is in Traditional Chinese, EXCEPT Firefox browser
> is in Japanese screenshot-3
>
> One new bug here. originally, I saved the screenshot file on the desktop. I moved the file on the screen, it suddenly disappeared. I checked the desktop folder, the screenshot files are still there.
> screenshot-4
> screenshot-5: locale and other information provided.
>
> 6. move English (United States) to the first one in the language options.
> Reboot the system. the language of menu on the top panel is still in Traditional Chinese, EXCEPT Firefox browser is in Japanese. Strange, all the files in desktop folder are showed again.
> screenshot-6
> screenshot-7: locale and other information provided.
>
> 7. reboot one more time. no change. move move English after English (United States) in the language options
> screenshot-8
>
> 8. reboot system. the language of menu on the top panel is in English.
> however the folder name is in Traditional chinese. Usually system
> shoudl pop up a screen which ask users if they want to keep the old
> name, but no this time.
>

I believe all this should be related to the issue described in bug
553162 and the fact that some programs, most notably Firefox, ignore the
LANGUAGE value and use LANG instead.
--
David Planella
Ubuntu Translations Coordinator
david(dot)planella(at)ubuntu(dot)com
www.ubuntu.com

[khuang (khuang)]

I'm experiencing this problem on 10.04 Final AMD64. Clean install.

Changing language in GDM does not affect anything except for Firefox and the Home folder names.

[asala (asala)]

I am also experiencing this on 64 bit 10.04 final, upgrading from 9.10.
Lots of perl "locale" errors on the terminal while upgrading... then,
gnome-language-selector unusable (window disappeared half a second after launch), until I changed (I think) export LC_ALL=C, did sudo dpkg-reconfigure locales
and then managed to run gnome-language-selector.
Anyway, now EVERYTHING (gnome menus, calendar, firefox, bash window, ...) is in English except a handful of applications ("update manager", "about gnome") which are half-english half Spanish.
>echo $LANG
ca_ES.utf8

>echo $LANGUAGE
es_ES:ca:es:en_GB:en

>echo $GDM_LANG
ca_ES.utf8

>cat .dmrc

[Desktop]
Language=ca_ES.utf8
Layout=es

and gnome menus in English (Appliations, Places, Systems) keep appearing whatever language I choose in the login screen.
This is a show-stopper for people which does not speak English (my father is happy with ubuntu 9.10 but I don't dare to upgrade should he lose Spanish support): an operating system which does not speak your language is almost as useless as one that hangs.

[asala (asala)]

Please don't interpret my last sentence despectively... I was thinking in "ubuntu for the masses" and what a not-so-cultivated non-English user would think of Ubuntu's reputation if this bug hits him. My experience with Ubuntu 9.10 was great.

[Martin Pitt (pitti)]

This was just discussed in #ubuntu-desktop. The basic problem here is that language-selector now is able to set system-wide locale settings which gdm does not understand. gdm is only able to set a locale (i. e. $LANG), while language-selector can set a locale and additionally a language list (which overrides LC_MESSAGES from $LANG) in $LANGUAGE.

I think a reasonable compromise would be:

 * As long as the locale picked in gdm matches the system locale, keep $LANGUAGE, since it's explicit configuration that the admin did for this machine, and we need to respect it. (cf. bug 407300)

 * Once the user selects a different locale, $LANGUAGES should be unset. This can be one of:

    - $LANGUAGES is set system-wide, and $GDM_LANG != the system wide locale
    - $LANGUAGES is set in ~/.profile, and $GDM_LANG != $LANG set from ~/.profile or ~/.dmrc

This needs to be done in /etc/gdm/Xsession.

[Fumihito YOSHIDA (hito)]

This bug has yet another affection of i18n breakage,

 - GDM read ~/.dmrc 's "Language" value and set LANG/LANGUAGEenvironment varibale.
 - ~/.dmrc has "Language=(langname).utf8" form by language-selector in some cases.
    (such as Lucid clean install without "no touch" boot. after install, ~/.dmrc has ".utf8" values)
 - So, LANG/LANGUAGE environment varibales sets "Language=(langname).utf8" by GDM.

But, LANG/LANGUAGE environment varibales expected 'LANG="(langname).UTF-8"' style, not "(langname).utf8".

This LANG/LANGUAGE settings behavior brakes some i18n codepages related applications (such as xarchiver, file-roller with archiver that has "codepage handling").

We must recover(or mitigate) dmrc's "Language" value too.

[Martin Pitt (pitti)]

Fumihito YOSHIDA [2010-05-11 12:51 -0000]:
> But, LANG/LANGUAGE environment varibales expected
> 'LANG="(langname).UTF-8"' style, not "(langname).utf8".

That's not true. ".utf8" is the canonical name now, and .UTF-8 is an
alias.

> This LANG/LANGUAGE settings behavior brakes some i18n codepages related
> applications (such as xarchiver, file-roller with archiver that has
> "codepage handling").

These would be bugs in those applications then, not gdm.

Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Peter M. Clausen (pclausen)]

hmm, I just installed German support for my girlfriend under her user. Then, logging off, changing to my user acount choosing "English" as a language (german keyboard layout) everything stays german.

clausen@clausen-ubuntu1004:~$ echo $LANG
en_GB.utf8
clausen@clausen-ubuntu1004:~$ echo $LANGUAGE
de_DE:de:en_GB:en
clausen@clausen-ubuntu1004:~$ echo $LANG
en_GB.utf8
clausen@clausen-ubuntu1004:~$ echo $LANGUAGE
de_DE:de:en_GB:en
clausen@clausen-ubuntu1004:~$ cat ~/.dmrc

[Desktop]
Language=en_GB.utf8
Layout=de
clausen@clausen-ubuntu1004:~$ cat /var/cache/gdm/$USER/dmrc

[Desktop]
Language=en_GB.utf8
Layout=de

[Arjan van Leeuwen (avleeuwen)]

> That's not true. ".utf8" is the canonical name now, and .UTF-8 is an
> alias.
>
>> This LANG/LANGUAGE settings behavior brakes some i18n codepages related
>> applications (such as xarchiver, file-roller with archiver that has
>> "codepage handling").
>
> These would be bugs in those applications then, not gdm.

We have a similar problem at Opera with Ubuntu users, they are filing reports with us that their locale doesn't work if they use gdm.

All these applications - including Opera - call standard X11 functions to set X's locale (see XSupportsLocale(3)). The problem is that some locales in the *.utf8 form are not supported by X11 as distributed with Ubuntu, but their *.UTF-8 counterparts are. An example of this would be bg_BG.utf8, which is one of the locales that gdm will set.

This can be fixed in two ways: gdm should use *.UTF-8 instead (as kdm does on Kubuntu), or X11 should be fixed to support the various *.utf8 locales that are missing (this can be done by adding the locale to X11's locale.alias file).

I don't have an exhaustive list of locales that don't work - I only know about bg_BG.utf8 and ru_UA.utf8 from bug reports by our users - but in the specific case of bg_BG.utf8 it's worth noting that that entry is in fact present in locale.alias on Ubuntu, but misspelled as be_BG.utf8. There is no entry for ru_UA.utf8.

[Sebastien Bacher (seb128)]

The bug has been assigned to the canonical desktop team for a while without change, pitti do you know want to work on this when you have a free slot or should be just unassign the bug for now?

[Kevin Huang (wasikevin)]

Without change means no progress.

Kevin Huang, Canonical

On 2010/8/3, at 19:42, Sebastien Bacher <email address hidden> wrote:

> The bug has been assigned to the canonical desktop team for a while
> without change, pitti do you know want to work on this when you have a
> free slot or should be just unassign the bug for now?
>
> ** Changed in: gdm (Ubuntu)
> Assignee: Canonical Desktop Team (canonical-desktop-team) => Martin Pitt (pitti)
>
> --
> Unset $LANGUAGES if the user picks a different locale in gdm, so that language-selector and gdm stop disagreeing
> https://bugs.launchpad.net/bugs/553162
> You received this bug notification because you are a direct subscriber
> of the bug.

[David Planella (dpm)]

Just for the record, this seems to affect Kubuntu as well. I've had at least two reports on IRC from people with different languages in LANGUAGE and LANG, with a half translated system (one in one language and one in the other).

There are also some people who install Ubuntu in English and then simply change the the language, and are affected by this. The first place where they notice is on Firefox, which does not follow LANGUAGE (bug 539761) and then appears in English.

Due to the number of duplicates, the ease to reproduce it and the usability impact, would it be possible to raise the importance of this one?

Thanks!

[Sergio Zanchetta (primes2h)]

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

[Sergio Zanchetta (primes2h)]

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

raising the importance.

[Sergio Zanchetta (primes2h)]

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

I raise the importance of the bug.

[Gunnar Hjalmarsson (gunnarhj)]

I believe that the problems that are addressed in this bug mainly fall into either or both of two categories:

1. - The LANGUAGE and LANG env. variables are often mixed up. LANGUAGE is meant to control the display language, while LANG is meant to control other locale aspects such as date formats etc.

2. - Changes in language-selector of LANGUAGE get overridden by gdm.

I have uploaded two branches to Launchpad and proposed that they are merged into lp:ubuntu/gdm respektive lp:ubuntu/lucid-updates/language-selector. Related patches that take care of both problem categories have been attached to this bug.

We also have the Firefox related problem, described both here and in LP: #550222. I added a patch to the latter that makes Firefox behave like one would expect.

Actually, with these three patches implemented I can seamlessly switch languages and other locales back and forth without them interfering with each other. Hopefully those of you who test the patches will make the same experience.

[Gunnar Hjalmarsson (gunnarhj)]

Can't see how unsetting $LANGUAGE would not create new problems, so I changed the summary.

[Martin Pitt (pitti)]

Gunnar,

Making up a $LANGUAGE from $LANG is hard to impossible. I think we should just unset it, as per comment 10. Also, the gdm script should not _ever_ modify files aside from ~/.wmrc, in particular not the user's profile. The current one doesn't take into account different syntax variants for setting and exporting the variable, and it can't do anything about setting it from /etc/default/locale anyway.

[Gunnar Hjalmarsson (gunnarhj)]

@Martin
Well, I don't consider it to be a problem, as you say in comment #10, that language-selector distinguishes between language and other locale aspects. That's just as it should be IMO. As far as I understand, the compromise you currently advocate would mean that Ubuntu's localisation model would keep being inconsistent, and an annoyance for e.g. people who prefer German locale settings while messages are displayed in English. Not to mention people who often switch language and/or locale settings...

We need to take into account how the different packages work together. language-selector (via LanguageSelector.py) modifies both ~/.profile and /etc/default/locale already.

I really think that this issue deserves a thorough discussion. To start with, it would be great if you could test the two debdiffs I attached to this bug report. Until somebody has proved me wrong, I believe that it is possible, and that I have figured out a good way to make gdm and language-selector work seamlessly together. :)

Btw, if you want to see that it also can work with Firefox, please see the solution I provide in bug 550222.

[Martin Pitt (pitti)]

I don't see why it should be inconsistent. If you pick a new language in gdm, then presumably you actually want to use this language (why else did you do it?), so in this case we should really empty $LANGUAGE. We just need to care about leaving $LANGUAGE alone if the selected language in gdm already matches the one that the user has in his profile, in other words, when he did not change it.

[YunQiang Su (wzssyqa)]

for example user's LANGUAGE is set zh_CN:zh_TW:zh_HK:en_US:en_GB

and in gdm he/her set to en_GB

can we do it like this:

gdm set LANGUAGE to

en_GB:zh_CN:zh_TW:zh_HK:en_US

or even

en_GB:zh_CN:zh_TW:zh_HK:en_US:en_GB

it is simple and can work.

[Gunnar Hjalmarsson (gunnarhj)]

@ Martin
language-selector sets $LANGUAGE for language, and $LANG for other locales. That's why it's inconsistent that gdm sometimes uses $LANG when dealing with the language to be used for message display.

I have attached a modified gdm-patch. These are the news:

- As you pointed out in comment #24, the previous code didn't take into account different syntax variants for setting and exporting $LANGUAGE. Now the code for editing ~/.profile is safer.

- The $LANGUAGE priority list is preserved, as was suggested in comment #27 by YunQiang Su.

- $LC_MESSAGES is set to take care of applications that don't recognize $LANGUAGE.

The third item seems to solve reported problems with Mozilla apps (Firefox/Thunderbird), which means that the solution I provided at bug 550222 already can be regarded as superseded. Also other programs that apparently don't recognize $LANGUAGE, e.g. FileZilla and Cervisia, now display menues etc. in the expected language.

Personally I think we should give high priority to this issue, and try to attain consensus about a solution. I'd be happy to contribute to achieve that goal ( seems like I have started... :) ). I noticed at ubuntu-devel that David Planella asked for suggestions for translation focus aspects with an eye to 11.04. Wouldn't a consensus solution to this issue be a suitable item on the list that David is preparing at https://wiki.ubuntu.com/Translations/Roadmaps/11.04 ?

[David Planella (dpm)]

Hi Gunnar,

Thanks for your work on this. I'll let Martin comment on the technical
aspects and add only a couple of comments.

El dl 04 de 10 de 2010 a les 08:38 +0000, en/na Gunnar Hjalmarsson va
escriure:
> @ Martin
> language-selector sets $LANGUAGE for language, and $LANG for other
> locales.

Language selector sets $LANGUAGE to use the possibility of setting
fallback languages. I haven't looked at the code to see if it sets $LANG
as well or it leaves it to gdm to do it, but it does allow setting the
LC_* categories individually.

> That's why it's inconsistent that gdm sometimes uses $LANG when
> dealing with the language to be used for message display.
>

GDM does not currently write $LANGUAGE.

> I have attached a modified gdm-patch. These are the news:
>
> - As you pointed out in comment #24, the previous code didn't take into
> account different syntax variants for setting and exporting $LANGUAGE.
> Now the code for editing ~/.profile is safer.
>
> - The $LANGUAGE priority list is preserved, as was suggested in comment
> #27 by YunQiang Su.
>
> - $LC_MESSAGES is set to take care of applications that don't recognize
> $LANGUAGE.
>

For the messages' locale, after evaluating LANGUAGE, the order of
evaluation in setlocale() is LC_ALL, LC_MESSAGES, LANG. So setting LANG
as gdm and language-selector do, should be enough. The problems I've
observed in Firefox, OpenOffice.org and other applications were related
to the fact that they did not support reading LANGUAGE. Were they not
reading LANG, either?

Rather than using a workaround, would it not be better to fix the bug in
the affected applications? I see you've followed up on Firefox already.

> The third item seems to solve reported problems with Mozilla apps
> (Firefox/Thunderbird), which means that the solution I provided at bug
> 550222 already can be regarded as superseded. Also other programs that
> apparently don't recognize $LANGUAGE, e.g. FileZilla and Cervisia, now
> display menues etc. in the expected language.
>
> Personally I think we should give high priority to this issue, and try
> to attain consensus about a solution. I'd be happy to contribute to
> achieve that goal ( seems like I have started... :) ). I noticed at
> ubuntu-devel that David Planella asked for suggestions for translation
> focus aspects with an eye to 11.04. Wouldn't a consensus solution to
> this issue be a suitable item on the list that David is preparing at
> https://wiki.ubuntu.com/Translations/Roadmaps/11.04 ?
>

As I noted on the follow up e-mail and on the blog post, that roadmap is
for translations community plans. It is not thought as a feature list
for Launchpad or a list of bugs to fix.

However, you should feel free to continue your work on this and address
the maintainer's comments. Keep up the good work!

Regards,
David.
--
David Planella
Ubuntu Translations Coordinator
www.ubuntu.com / www.davidplanella.wordpress.com
www.identi.ca/dplanella / www.twitter.com/dplanella

[Gunnar Hjalmarsson (gunnarhj)]

@ David
Thanks for your praising comment.

It seems to me that you, and possibly also Martin, assume that $LANG ought to correspond to the first language in the $LANGUAGE list. That, I believe, is a misconception, at least from a language-selector perspective, and I fear that we have talked at cross-purposes so far.

In an attempt to help changing that, I wrote a wiki-page:
https://wiki.ubuntu.com/GdmLanguageSelectorDissonance

It summarizes my view on where we stand, where we want to go ("Desired behavior"), and my proposed solution in English. It would be great if you, Martin and anybody else with an interest in this issue could study the page and add your possible objections and comments.

Basically, what I hope to convince you about is that since it's only language that the users can set from GDM, and not any other locale aspects, GDM should not do anything with the $LANG variable. Not at any point. Never.
Instead GDM should update $LANGUAGE and set $LC_MESSAGES. That appears to be sufficient to reliably control which language is used in menus and messages. $LANG is used for the non-translation locale aspects, and its value shall not be used as a fall back by programs that don't understand $LANGUAGE.

As regards apps that don't understand $LANGUAGE:
> Rather than using a workaround, would it not be better to fix the
> bug in the affected applications? I see you've followed up on
> Firefox already.

If I have understood it correctly, $LANGUAGE is a gettext specific variable, and certain applications' inability to understand that variable is not a bug per se. Consequently I think that Ubuntu in any case shall provide the preferred translation language also via the $LC_MESSAGES variable, and I don't think that's a workaround. Rather should the Firefox fix I provide at bug 550222 be considered a workaround IMO.

I noted that you don't consider this issue a matter for the translation community. But it's a borderline case, isn't it? ;-)

[Martin Pitt (pitti)]

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-04 8:38 -0000]:
> language-selector sets $LANGUAGE for language, and $LANG for other
> locales. That's why it's inconsistent that gdm sometimes uses $LANG
> when dealing with the language to be used for message display.

gdm doesn't have such an elaborate locale/language configuration as
the language-selector, it just as a simple locale selector. It simply
doesn't support setting a list of language fallbacks for $LANGUAGE.
It's not supposed to either, the main point is that you can start a
desktop session in a different language. You can then fine-tune it
with language-selector.

Thus gdm is not really inconsistent, it just doesn't have feature
parity with language-selector.

Therefore it should not try to change your $LANGUAGE settings either.
It should just clear it when it changes $LANG, so that $LANG actually
becomes effective.

> - As you pointed out in comment #24, the previous code didn't take into
> account different syntax variants for setting and exporting $LANGUAGE.
> Now the code for editing ~/.profile is safer.

No, please let's not. gdm is _not_ a tool for setting $LANGUAGES.
That's what language-selector is for.

> - $LC_MESSAGES is set to take care of applications that don't recognize
> $LANGUAGE.

That should be fixed in language-selector, not gdm. gdm already sets
$LANG, which firefox & friends do recognize.

Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Martin Pitt (pitti)]

Gunnar Hjalmarsson [2010-10-05 4:13 -0000]:
> It seems to me that you, and possibly also Martin, assume that $LANG
> ought to correspond to the first language in the $LANGUAGE list.

No, it shouldn't. I wasn't saying that, I was just saying that, since
gdm doesn't have a way to configure $LANGUAGES, there is no point in
faking $LANGUAGES, _because_ $LANG and $LANGUAGES are independent.

> Basically, what I hope to convince you about is that since it's only
> language that the users can set from GDM, and not any other locale
> aspects

No, it's exactly the other way around. gdm has a locale selector, but
not a language list selector.

> If I have understood it correctly, $LANGUAGE is a gettext specific
> variable, and certain applications' inability to understand that
> variable is not a bug per se.

That's correct, $LANGUAGE is a GNU extension.

> Consequently I think that Ubuntu in any case shall provide the
> preferred translation language also via the $LC_MESSAGES variable,
> and I don't think that's a workaround.

I agree, this should indeed be fixed in language-selector.

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Gunnar Hjalmarsson (gunnarhj)]

Martin,

On 2010-10-07 11:11, Martin Pitt wrote:
> [gdm] just has a simple locale selector ... the main point is that
> you can start a desktop session in a different language.

That's crystal clear to me. And as long as language-selector didn't distinguish between language and other locale aspects, the natural way to serve that purpose was to let GDM set $LANG. There was no reason to do it otherwise.

As from Lucid (revision 79 in language-selector, I think) language-selector works differently, and currently it does not play well with GDM. Making GDM and language-selector work well together in this respect, i.e. preventing unexpected behavior and user confusion, is the focus I have. Please note that I suggest changes to both language-selector and GDM.

Keeping the limited purpose of GDM's locale selector in mind, i.e. allowing users to start a session in a different language, I believe that _now_ the best way to serve that purpose is to update $LANGUAGE and set $LC_MESSAGES, while leaving $LANG to language-selector. In short: Keep the purpose, change the method.

So, why do I persist, and claim that my method is better? As far as I can tell, the patches I wrote make language-selector and GDM work seamlessly together, without any cause left for users to get confused. At the same time I can't see how going the other way, and just unset $LANGUAGE, would be sufficient to prevent undesired user surprises.

In an attempt to make my point I added this code to Xsession:

    if [ $( echo $GDM_LANG | sed 's/\..*//' ) != \
         $( echo $LANGUAGE | sed 's/:.*//' ) ]; then
        unset LANGUAGE
    fi

Then I made sure that everything was set to Swedish to start with. Now, please consider this scenario:

* I decide to switch the language (and nothing else) to en_US, and do so from GDM.

* While now seeing English in menus and messages, I also see those ambigous American date formats etc., which I don't want.

* Without actually understanding what happened, I go to Language Support and open the tab for non-language locale settings. To my surprise, it appears as if Swedish is still selected. Weird! Maybe a temporary glitch...?

* I restart the computer. No change.

* This time, when in the Language Support tab for non-language locale settings, I change (from Swedish) to something else, and then back to Swedish. (I happen to know by now that that makes a difference, but does the average user know? Probably not.)

* Restart.

* Now Swedish is the pre-selected _language_ in GDM. Seems like we are back at square one. :(

This is certainly not what I want, and I'm sure that you do not want it like that either.

So, what else can I say? Suppose most that can be said have been said by now, so I simply beg you to reconsider and give my patches a chance. If I'm proved wrong, by you or somebody else, I promise that I'll shut my mouth. But only then. ;-)

[Martin Pitt (pitti)]

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-07 22:34 -0000]:
> Keeping the limited purpose of GDM's locale selector in mind, i.e.
> allowing users to start a session in a different language, I believe
> that _now_ the best way to serve that purpose is to update $LANGUAGE and
> set $LC_MESSAGES, while leaving $LANG to language-selector.

If gdm's UI becomes a pure language selector, I agree. This requires
some more intrusive code changes, though, and some thought about
handling country specific languages like Portugese from Portugal vs.
Brazil, or simplified/traditional Chinese. This should be
coordinated/discussed in an upstream bug, since this is going to be a
fairly large patch.

> So, why do I persist, and claim that my method is better? As far as I
> can tell, the patches I wrote make language-selector and GDM work
> seamlessly together, without any cause left for users to get confused.

Your gdm patch just changes how the result from the locale selector is
handled, but it doesn't change the locale selector to become a
language selector. So if I'm choosing "German (Switzerland)" on an
en_US-by-default box, I'd get German strings, but time/currency/etc
format would still be US. This would be by design with your idea (set
LANGUAGE=de and keep LANG=en_US), but gdm shouldn't offer me three
different country options for Germany, since they won't make a
difference.

> At the same time I can't see how going the other way, and just unset
> $LANGUAGE, would be sufficient to prevent undesired user surprises.

In my scenario above, unsetting $LANGUAGE when I change LANG away from
the default en_US, would ensure that I actually get a fully German
desktop.

> * I decide to switch the language (and nothing else) to en_US, and do so
> from GDM.

You can't right now. en_US is not a language, it's a locale.

> * While now seeing English in menus and messages, I also see those
> ambigous American date formats etc., which I don't want.

Right, I understand (see above). That's exactly my point above, gdm
shouldn't offer a country selection if it stops being able to set
country specific settings.

> So, what else can I say? Suppose most that can be said have been said by
> now, so I simply beg you to reconsider and give my patches a chance. If
> I'm proved wrong, by you or somebody else, I promise that I'll shut my
> mouth. But only then. ;-)

Oh, please don't misunderstand me. I don't think your idea is wrong,
it just changes some use cases. However, if we want to implement it,
it should be done properly, not just halfway (and thus become even
more inconsistent).

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Gunnar Hjalmarsson (gunnarhj)]

Hi again, Martin!

On 2010-10-08 13:08, Martin Pitt wrote:
> Gunnar Hjalmarsson [2010-10-07 22:34 -0000]:
>> Keeping the limited purpose of GDM's locale selector in mind, i.e.
>> allowing users to start a session in a different language, I believe
>> that _now_ the best way to serve that purpose is to update $LANGUAGE and
>> set $LC_MESSAGES, while leaving $LANG to language-selector.
>
> If gdm's UI becomes a pure language selector, I agree. This requires
> some more intrusive code changes, though, and some thought about
> handling country specific languages like Portugese from Portugal vs.
> Brazil, or simplified/traditional Chinese.

I thought that GDM was already prepared for handling region specific languages (see below).

> Your gdm patch just changes how the result from the locale selector is
> handled, but it doesn't change the locale selector to become a
> language selector. So if I'm choosing "German (Switzerland)" on an
> en_US-by-default box, I'd get German strings, but time/currency/etc
> format would still be US. This would be by design with your idea

So far I'm with you (I think...).

> (set LANGUAGE=de and keep LANG=en_US),

Not true as regards LANGUAGE. If you would choose "German (Switzerland)", my patch would make de_CH, i.e. including the country code, become the first language/country combination in the LANGUAGE list. (It would also set LC_MESSAGES to "de_CH.utf8".)

Please note that the LANGUAGE list, that is currently maintained from language-selector only, includes country codes when available. Consequently the code I patched does so too.

> but gdm shouldn't offer me three different country options for Germany,

Why not? Ok, it should offer you six different country options, because that's the available number. ;-)

> since they won't make a difference.

They will (se above). As regards translated strings, that is. Not as regards other locale aspects.

While I agree that GDM's UI should be reconsidered if a change along these lines is implemented, I'm not convinced, at least not yet, that there is actually a need to change it.

>> * While now seeing English in menus and messages, I also see those
>> ambigous American date formats etc., which I don't want.
>
> Right, I understand (see above). That's exactly my point above, gdm
> shouldn't offer a country selection if it stops being able to set
> country specific settings.

Well, with my patch it _would_ be able to set a country/region specific language. Personally I think that the GDM UI explains it: The label is "Language", and the list includes region specific options.

Or do you mean that users are currently led to believe that GDM changes _all_ locale settings? Personally I don't think they are, possibly with the exception of experienced Ubuntu users, who may remember how it used to work...

> Oh, please don't misunderstand me. I don't think your idea is wrong,
> it just changes some use cases. However, if we want to implement it,
> it should be done properly, not just halfway (and thus become even
> more inconsistent).

Absolutely.

Let me say that I like the direction this conversation now has taken, and I'd be happy to keep participating in the process. As a n...

[Hsin-Yi, Chen (hychen) (ossug-hychen)]

I agree with Gunnar Hjalmarsson, if a user changes the language in GDM, it should only change the interface language(set LANGUAGE, LC_MESSAGE variable), not ALL locale seetings.

As a Taiwan user, the default display language after Ubuntu installed is Traditional Chinese, but I prefer my interface message displays in English and keep the currency, datetime, formate, input method is zh_TW.

To have language-selector and location-selector option in GDM would be better.

I believe this requirement is also needed by foreigns live in Taiwan. (who may are married with Taiwanese or are studying in Taiwan school.)

[August Karlstrom (fusionfile)]

I can't understand how the developers or testers of the language selector have missed the very basic test scenario of choosing different languages for text and numbert etc. and then starting the most commonly used application: Firefox.

[Gunnar Hjalmarsson (gunnarhj)]

Some additional thoughts.

Irrespective of which changes we agree upon at last, considering the high importance of this issue they ought to be implemented, and people given sufficient time to test them, in good time for the Natty release. To make testing easier, I created ppa:gunnarhj/locale-test and uploaded versions of the gdm and language-selector packages that include the code changes I suggest. It means that we now can test the solution by just installing the binary files (provided that we upgrade to Ubuntu 10.10). Example:

    sudo dpkg -i language-selector_0.6.6localefix1_all.deb

Martin, you mentioned an upstream bug, because of the assumed size of the GDM patch. As regards a significant GDM UI change, I still don't believe it will be necessary. I believe that we are just about to turn GDM's locale selector into a country/region specific language selector. (I took pains to clarifying my view when I formulated the PPA changelog entries.)

Problably _some_ remaining code tweaking will prove to be motivated, and I'll be happy to help with it. But wouldn't it be better if our further discussion and testing take place concurrently?

/ Gunnar

[Gunnar Hjalmarsson (gunnarhj)]

Hmm.. Thought that Launchpad would make a link from ppa:gunnarhj/locale-test. Apparently not.

This is the address to the PPA: https://launchpad.net/~gunnarhj/+archive/locale-test

[Martin Pitt (pitti)]

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-08 13:55 -0000]:
> > but gdm shouldn't offer me three different country options for
> Germany,
>
> Why not? Ok, it should offer you six different country options, because
> that's the available number. ;-)

I mainly mentioned this because for many languages there isn't
actually much difference. E. g. for German, Launchpad/Ubuntu langpacks
just ship "de", since the variations in the different countries are
negligible and just lead to bloat or inconsistent translations. The
major exceptions are Portugese, Chinese, and British English.

So if we turn this into a pure language selector, it could/should
become much smaller. However, I agree that this is only a secondary
issue.

> Or do you mean that users are currently led to believe that GDM changes
> _all_ locale settings? Personally I don't think they are, possibly with
> the exception of experienced Ubuntu users, who may remember how it used
> to work...

It's what gdm has been so far. TBH I don't know how many people would
expect that this also changes the time/date/currency/etc. format for
them, but I don't think this expectation is unreasonable, especially
not if you can pick a country in the list.

So, after this discussion I'm not opposed to turning gdm into a pure
language selector, and ask users to use language-selector (ugh, how
confusing :) ) to setup the full locale. But if we do that, I'd like
it to be done properly, i. e. upstream, and have gdm set $GDM_LANG
without requiring extra hacks like sed 's/^en\./en_US./' (remember
that this stuff runs on every boot and costs precious boot time).

Also, changing ~/.profile in Xsession.in is still an absolute no-go
for me. gdm writes its settings to ~/.dmrc, and that's where it should
live. grepping/testing/writing ~/.profile at each boot is just wrong.
I hope I'll be able to convince you about this point, then we'd have
consensus. :-)

Thanks, and have a nice evening,

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Martin Pitt (pitti)]

Gunnar Hjalmarsson [2010-12-02 7:24 -0000]:
> Yeah, but personally I believe there is still a need for _one_ responsible
> team leader.

For the record: Indeed l-s currently doesn't have a lead developer,
but we'll get one soon.

Right now it's on collaborative maintenance mode. (There are some
changes from me in bzr as well)

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Martin Pitt (pitti)]

Hello Gunnar,

Gunnar Hjalmarsson [2010-12-07 8:42 -0000]:
> * Patched the changes to Xsession.in, gdm-session-settings.c and
> gdm-session-settings.h. Sub-scripts belonging to the
> language-selector package dropped - yes, I changed my mind. ;-)

Oh, great :)

> Natty problem:
> When I uploaded the GDM source to my ppa, it failed to build for Natty.
> https://launchpad.net/~gunnarhj/+archive/locale-test/+build/2083108

I see. This seems to be unrelated to your changes, so I guess I'll fix
that first in our ubuntu-desktop gdm branch, and then apply the merge
on top of this.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Martin Pitt (pitti)]

Unsubscribing sponsors, for the record. I think the discussion here has become way too complex, this will be handled between Gunnar and me now.

[Gunnar Hjalmarsson (gunnarhj)]

Hi Martin,

Thanks for your comments! Before replying to some of them, let me say
that I finally realized that the .d/ sub-scripts solution I suggested
previously was too obfuscated. Whatever advantages I saw, they could not
justify the resulting obfuscation. All code in a project like this has
to be reasonably easy to understand and maintain.

I'm not sure, but judging from some of your comments, I have a feeling
that even you had some difficulties in seeing through what the code
actually did. My fault, in that case; sorry.

Besides my replies below, I have added a comment to the GDM merge
proposal.

On 2010-12-07 10:34, Martin Pitt wrote:
> If you patch /etc/gdm/Xsession, it'll just affect gdm. If you add
> something to /etc/profile.d/ and xinitrc.d/, it will affect all
> login managers, and will even run for console and ssh logins.
> ...
> There's another aspect to this: The upstream gdm Xsession script
> already sets $LANG. With the changes that you have in mind, it needs
> to stop doing that; with separate .d directory scripts, you can only
> additionally set $LC_*, but the original $LANG value is lost.

I believe that I had taken care of both those aspects. The code in
/etc/profile.d/, which would have run before the spot where Xsession
sets $LANG, looked like this:

    if [ -n "$GDM_LANG" ]; then
        export GDM_LANGUAGE=$GDM_LANG
        unset GDM_LANG
    fi

The code in xinitrc.d/ would not have done anything but a variable test,
if [ -n "$GDM_LANGUAGE" ] had returned false.

Even if it's history now, I'd appreciate a hint if I missed anything.

> ... we need to have a compromise where we don't change the meaning of
> .dmrc. Since other DMs also use that file, we can't just redefine the
> meaning of the Language= option there anyway.

I didn't know that other DMs use the .dmrc file; thought it was a GDM
related file like the GDM_LANG variable. Hmm.. Spontaneously I don't
think it's an issue that hasn't already been taken care of, though,
since language-selector updates .dmrc only if GDM is running. Other
login managers can safely keep saving its settings in .dmrc.

I removed a section, where you further emphasize that we need to prevent
undesired and/or surprising effects to other login managers but GDM. On
that we are 100% agreed. My apologies if I have given you some other
impression.

Please note that the "redefinition" I suggest would only apply when GDM
and language-selector play together. Replace GDM, and l-s stops writing
to .dmrc. Remove l-s, and GDM falls back to its original behavior. No
harm done.

>> So, why doesn't language-selector pass the same string to dmrc as
>> it does to LC_MESSAGES? My reason for not wanting to do it that way
>> is that it would result in incorrect pre-selected options in GDM's
>> language picker. For instance, if you would drag "Deutch" to the
>> top of the menu in language-selector's language tab, you would see
>> "German (Austria)" as the pre-selected option at next login.
>> Certainly not a disastrous bug, but significant enough IMO to not
>> introduce intentionally.
>
> I agree that this is confusing, but I think that's the kind of
> compromise we have to make if this is going to be a ...

[Cyprian Guerra (cyprian-guerra)]

It gives me SO much headache. I understand what I propose is a separate approach but can someone please specify what drawbacks lie behind creating a "language-creator" instead of "language-selector"? There are only 10 types of users: those finding default language settings for their country convenient and those needing custom locale files. So why there's no creator that would allow to specify system messages language, date in a form of %Y-%M-%D or YYYY-MM-DD, put in a decimal separator, currency name, first day of the week and so on? It would auto-normalize after a time by users' statistics (e.g. it can automatically sent statistics after a few days without changes to the locales and after the user agrees) so every popular scheme would "come up" shortly and could be coming predefined with new system distributions. Why do I have to trying out all the locales one by one to find out where's the one with convenient date, or decimal separator, with learning whole lang file structure and creating one myself as the only alternative? Language-selector, when fixed, will only allow to speed-up the "trying out" part but after trying every default set I will still end up editing files I do not understand. Your approach gives only N/M-1 probability of satisfying the user, where N is the number of prepared sets and M is the number of possible language/country combinations. And those are small chances.

[Martin Pitt (pitti)]

For the record, current ubuntu-desktop gdm bzr head (and the version in natty) build fine again.

[Martin Pitt (pitti)]

Hello Gunnar,

Gunnar Hjalmarsson [2010-12-09 1:49 -0000]:
> > There's another aspect to this: The upstream gdm Xsession script
> > already sets $LANG. With the changes that you have in mind, it needs
> > to stop doing that; with separate .d directory scripts, you can only
> > additionally set $LC_*, but the original $LANG value is lost.
>
> I believe that I had taken care of both those aspects. The code in
> /etc/profile.d/, which would have run before the spot where Xsession
> sets $LANG, looked like this:
>
> if [ -n "$GDM_LANG" ]; then
> export GDM_LANGUAGE=$GDM_LANG
> unset GDM_LANG
> fi

Ah, I haven't paid attention to this bit before (I didn't review the
code in that detail yet, just took a look at the structure so far).
That would certainly have done the trick, although it's quite
non-obvious to someone who just reads the /etc/gdm/Xsession script.

> I didn't know that other DMs use the .dmrc file

At least lxdm and kdm do, I didn't check/know about others.

But exactly because it's a shared config file, I'm not too worried
about adding new fields to it. DMs should just ignore fields which
they can't deal with, and the .ini file format is meant for this. We
just need to pay attention to not change the semantics of existing
fields, which is why I'm so insistant on keeping Language= as a valid
locale.

> > I'd rather add a new LanguageList (or similar) field to it on
> > demand.
>
> Actually I played with that thought when preparing the current branches,
> but concluded that there is no need. Convince me that I'm wrong, and
> I'll be happy to do some copy-n-pasting again. ;-)

Oh, don't get me wrong -- if we can make-do with the existing Language
field, so much the better! I just had the impression that we also
wanted to store a list of languages for the $LANGUAGE variable, in
which case we'd probably need a new field.

> Sounds reasonable to me. Is there a more appropriate place than bug
> reports to hold this kind of complex (at least for a newbie like me)
> design discussions?

It's good enough IMHO. Well, an even better one is being in a room
together with a whiteboard :)

It just became too complex for a regular sponsor to pick up.

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Launchpad Janitor (janitor)]

This bug was fixed in the package language-selector - 0.8

---------------
language-selector (0.8) natty; urgency=low

  * Set LC_MESSAGES for applications that don't recognize LANGUAGE
    (LP: #553162).
  * GDM related fixes (LP: #553162):
    - Update dmrc when LANGUAGE and LC_MESSAGES are set, not when LANG
      is set.
    - Use the new dmrc fields "Langlist" and "LCMess" to store the
      LANGUAGE and LC_MESSAGES values on disk.
 -- Gunnar Hjalmarsson <email address hidden> Tue, 14 Dec 2010 22:20:37 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package gdm - 2.32.0-0ubuntu2

---------------
gdm (2.32.0-0ubuntu2) natty; urgency=low

  [ Martin Pitt ]
  * 06_run_xsession.d.patch: Don't trip over directories and other non-files
    in Xsession.d/. (LP: #654578)

  [ Gunnar Hjalmarsson ]
  * debian/patches/35_langlist_and_lsmess_dmrc_fields.patch:
    - Addition of the fields "Langlist" and "LCMess", which make ~/.dmrc
      and /var/cache/gdm/$USER/dmrc able to store the user language
      environment (LP: #553162).
  * debian/patches/36_language_environment_settings.patch:
    - Changes to Xsession's way to use GDM_LANG. It now sets LANGUAGE
      and LC_MESSAGES, which makes it possible to keep the user language
      for message translation apart from other locale settings
      (LP: #553162).
 -- Martin Pitt <email address hidden> Tue, 14 Dec 2010 22:24:40 +0100

[YunQiang Su (wzssyqa)]

It costs almost One year !

--
YunQiang Su

[Gunnar Hjalmarsson (gunnarhj)]

@trespasser (comment #87)
Personally I would like to see a GUI where users can seemlessly set their locale preferences without knowing what a locale is, so I find your suggestion interesting. However, it's beyond the scope of this bug report. These are two places that may be more suitable to discuss your idea:

http://brainstorm.ubuntu.com/

https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

A wishlist bug might also be an option.

/ Gunnar

[Gunnar Hjalmarsson (gunnarhj)]

Hi all,

As you may have seen, fixes to this bug report were released in Natty on December 14, and yesterday Martin Pitt approved the last 'follow-up fix'.

The issues were introduced in Lucid (Ubuntu 10.04), so I have prepared gdm and language-selector packages for Lucid and Maverick with similar changes. For the time being they are available at https://launchpad.net/~gunnarhj/+archive/locale-test

Your feedback on the fixes would be much appreciated. For the case you would find buggy behavior that was discussed here, but has not been properly dealt with, please submit new, specific bugs. I for one picked up one such issue, and reported it as bug 693337.

/ Gunnar

[Martin Pitt (pitti)]

This introduces quite a substantial behaviour change, including the additional setting of $LC_MESSAGES (which now causes a lot of confusion when using ssh), has very intrusive patches, and this isn't a data loss or security bug, thus I am afraid it doesn't qualify for an SRU.

[Gunnar Hjalmarsson (gunnarhj)]

Ok Martin, no problem. You weren't very keen on the SRU idea in December either, but you mentioned backports as a possibly passable option. To me, the branches that this bug report resulted in are principally fixes of annoying, buggy behavior. However, I'm sure there are good reasons for the restrictive SRU criteria.

At https://help.ubuntu.com/community/UbuntuBackports I read:
"Please, for bugfixes, try https://wiki.ubuntu.com/StableReleaseUpdates first. We will reject any Backports requests for bugfixes if SRU has not been attempted."

I take it that I now can consider the SRU nomination in the making 'officially' rejected, and start checking out the backport process more closely. :-)

[Gunnar Hjalmarsson (gunnarhj)]

Hi all,

I have filed the bug 719815, where I request that the fixes of this bug be backported to Lucid and Maverick.

https://help.ubuntu.com/community/UbuntuBackports

Once test builds have been made, please feel free to help process the request by testing the packages and posting feedback and possible other comments on bug 719815.

[Gunnar Hjalmarsson (gunnarhj)]

Fixes of this bug for Lucid and Maverick are now available in official backports packages. To make Synaptic check for backports updates you can do:

o System -> Administration -> Update Manager -> Settings...

o Select the "Updates" tab and check the "Unsupported updates" option.

More about Ubuntu backports: https://help.ubuntu.com/community/UbuntuBackports

[Kevin Huang (wasikevin)]

re-tested GDM in Natty, and found the issues still remain.

languages pre-installed: English, Traditional Chinese, Simplified Chinese
GDM: 2.32.0-0ubuntu14

Reproduce steps

A. issue at GDM screen.
1. logout
2. select English on the bottom panel
3. Ideally, "密碼" should change to "password", but it remains no change. Please see the attached screen shot.

B.issue after login
1. logout
2. select ”漢語 台灣“ （Traditional Chinese） on the bottom panel
3. input password then login the system
4. the language keeps no changes as English

[Gunnar Hjalmarsson (gunnarhj)]

Kevin,

On 2011-04-05 11:02, Kevin Huang wrote:
> A. issue at GDM screen.
> 1. logout
> 2. select English on the bottom panel
> 3. Ideally, "密碼" should change to "password", but it remains no
> change.

That is as it should be. The language chooser on the bottom panel sets
the user language, while it's the system language that controls in which
language the labels on the login screen are displayed. Please use
System -> Administration -> Language Support
to change language system-wide, if that is what you want to do.

> B.issue after login
> 1. logout
> 2. select ”漢語 台灣“ （Traditional Chinese） on the bottom panel
> 3. input password then login the system
> 4. the language keeps no changes as English

If you select "Chinese (Taiwan)" or "Chinese (Hong Kong)" with the
language chooser on the login screen, and then log in, the session
indeed should be displayed in traditional Chinese. Did you really log
out first, or could it possibly be that you just returned to an
already existing session?

[Kevin Huang (wasikevin)]

Hi Gunnar,

let me reproduce it in details. Hope you don't mind to refer to I captured the screen shots in sequence attached, because I am not sure if I can explain it in details in English.

Environment
1. Fresh Unity Beta 1 installation with all updates on April 7th
2. Language selected in OOBE at installation: Simplified Chinese
3. GDM version: 2.32.0-0ubuntu15
4. Languages preinstalled: English, Simplified Chinese, Arabia (added after installation)

Testing steps

1. starting Locale: Simplified Chinese
2. Screen shot1 (bug): For some reason, Gwibber is in Aribic. Except Gwibber, other application names look fine.
3. log out
4. Screen shot2 (bug): The "Universal Access Preferences" icon on the right bottom is missing.
5. Screen shot3: password and language selection are in Simplified Chinese
6. Screen shot4: select language: English (America). The "password" is still kept as "密碼". From user's point of view, I would prefer it could be changed to "password" after I select language in English. However, I would say this is at lower priority.
7. login
8. Screen shot5: all names of preinstalled applications are in English. There are several applications in "Apps available for download" in Simplified Chinese.
9. logout
10. screen shot6: password and language selection are in Simplified Chinese which should be in English.
11. screen shot7: all languages in language selection list are in Simplified Chinese.
12. screen shot8: select language: 阿拉伯语 (Aribirc).
13: login
14: screen shot9: Most names of preinstalled application are in Arabic although the translation quality is not as high as in Simplified Chinese. There are several applications in "Apps available for download" in Simplified Chinese.
15: logout, select English, login, then install traditional Chinese and Japanese.
16: logout
17: no matter I select Japanese or Traditional Chinese, the GDM is always in Simplified Chinese, and several applications in "Apps available for download" are in Simplified Chinese.

[Gunnar Hjalmarsson (gunnarhj)]

Hi again, Kevin.

Two things you need to be aware about:

1. The display language can be set on two levels: system wide,
which determines what's shown on the login screen, and user specific.
It's only the user language that can be set from the login screen.

2. The principal tool for setting language is Language Support. The
language chooser on the login screen is just a supplement.

I have recently written a help document, which shows up if you click the
"Help" button in Language Support. Your latest comment triggered me to
make some changes to the document, but they are currently queued since
we are in BetaFreeze (https://wiki.ubuntu.com/BetaFreeze). You can still
reach those very latest changes by installing language-selector from my
PPA: https://launchpad.net/~gunnarhj/+archive/i18n-docs

I recommend you to study the document carefully, to get a better
understanding of how languages are handled in Ubuntu.

On 2011-04-08 05:13, Kevin Huang wrote:
> Hope you don't mind to refer to I captured the screen shots in
> sequence attached, because I am not sure if I can explain it in
> details in English.

Of course I don't mind. :) It's a good way to clarify your points and
prevent misunderstandings.

> 1. starting Locale: Simplified Chinese
> 2. Screen shot1 (bug): For some reason, Gwibber is in Aribic. Except
> Gwibber, other application names look fine.

Probably you either made Arabic be included in the language priority
list, or you picked an Arabic locale on the "Regional Formats" tab in
Language Support.

> 3. log out
> 4. Screen shot2 (bug): The "Universal Access Preferences" icon on the
> right bottom is missing.

I know that that bug has been addressed, and I'd be surprised if it's
not fixed before the 11.04 release.

> 5. Screen shot3: password and language selection are in Simplified
> Chinese
> 6. Screen shot4: select language: English (America). The "password"
> is still kept as "密碼". From user's point of view, I would prefer it
> could be changed to "password" after I select language in English.
> However, I would say this is at lower priority.

There is a wish for such a feature in bug 24935. Can't tell whether it
would be reasonable to implement it.

> 7. login
> 8. Screen shot5: all names of preinstalled applications are in
> English. There are several applications in "Apps available for
> download" in Simplified Chinese.

Probably those apps ignore both the LANGUAGE and LC_MESSAGES variables,
and use the $LANG value directly to determine the display language.

I have a vague idea of what might be done to improve Ubuntu's Language
Support in this respect, but it would need to be discussed first.

> 9. logout
> 10. screen shot6: password and language selection are in Simplified
> Chinese which should be in English.

No, it should not. Since you haven't changed the system language, it's
still simplified Chinese.

> 11. screen shot7: all languages in language selection list are in
> Simplified Chinese.

Since you haven't changed the system language, it's still simplified
Chinese.

> 12. screen shot8: select language: 阿拉伯语 (Aribirc).
> 13: login
> 14: screen shot9: Most names of preinstalled application are in
> Arabic although ...

[Kevin Huang (wasikevin)]

Hi Gunnar, Thanks for your clear explanation about two level setting of display language. Issue closed

[Gunnar Hjalmarsson (gunnarhj)]

You're welcome, Kevin. Let me thank you for your effort with the detailed questions. They helped me improve the new Language Support Help document.

Also, I'm glad that it wasn't yet another bug. ;-)

[Kentaro (kentarofukuchi)]

The patch #36 (36_language_environment_settings.patch) damaged my desktop environment by setting LANGUAGES.

I'm Japanese and my LANG is "ja_JP.UTF-8", but because of some reasons my LC_MESSAGES is set to "C" by .gnomerc and .zshenv to display non-translated messages/texts. This worked very well for me.

After upgrading to Natty, however, it went badly. Because GDM now sets LANGUAGES as "ja_JP:en_US:en" and this setting overrides LC_MESSAGES in many applications.

Mine is not a major requirement, but setting LANGUAGE is too strong and overriding various settings including LC_XXX.

[Kentaro (kentarofukuchi)]

Let me correct my previous comment:

s/LANGUAGES/LANGUAGE/g

[Gunnar Hjalmarsson (gunnarhj)]

@Kentaro
To have menus and messages displayed in English, just drag "English" to the top of the combobox list on the "Language" tab of Language Support.

[Kevin Huang (wasikevin)]

@Gunnar,

I guess there are very few users (not developers) know two level language set up designs, system wide on language support and user specific on the login screen. It actually has confused me for a while until I read your clear explanation: https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/553162/comments/102

[Kentaro (kentarofukuchi)]

@Gunnar, (#107)

Thank you for your help. I tested it and it seems to be working at this moment.

I was in doubt that selecting "English" of Language menu affects all locale settings such as LC_TIME or LC_NUMERIC.
Let me see what happens with my settings. Now I have "export LANG=ja_JP.UTF-8; export LC_MESSAGES=C" in my
.gnomerc and .zshenv. "echo $LANGUAGE" says "LANGUAGE=en_US:en". Gnome Clock displays current date in Japanese
format.

[Gunnar Hjalmarsson (gunnarhj)]

@Kevin
Yes, I remember that you were confused by system vs. user specific level. Language Support Help, which you can reach by clicking the "Help" button at the bottom left of the Language Support window, will hopefully contribute to reduce similar confusion henceforth. Also, at the UDS in Budapest it was decided to let a design team review the work flow and UI of Language Support.

Do you have a specific suggestion in mind?

@Kentaro
In Lucid and Maverick, choosing a locale for regional formats, that does not match the selected language, often leads to unexpected results, which I suppose that your special code in .gnomerc and .zshenv aims at correcting. In Natty that code ought to be redundant, since the problems have been fixed, so you should be able to remove the special code without any undesired behavior.

[dreamon (dreamon)]

I am not sure if this is related, but I am experiencing problems with GDM overriding language selections I apply on a per-programme basis via the env variable.

Normally, when I select Chinese (China) at the login screen, I can still run programmes in English by prepending an env argument, e.g. "env LANG=en_US.UTF-8 firefox". That way I can run an English instance of Firefox in a Chinese language Ubuntu session. This was working fine up to Lucid. Maverick was the first Ubuntu release that caused problems. After installing Maverick, whatever language I specified at the login screen would stick, and I couldn't run programmes in an alternative language by prepending env. The issue disappeared somewhere along the way after an update, though. However, I just finished setting up a fresh install of Natty and the problem reappaeared.

Is this related to this bug or described somewhere else? Has somebody else experienced similar problems?

[dreamon (dreamon)]

A screenshot demonstrating the conflict between gdm and env.

I selected "Chinese (China)" at the login screen to run a Chinese language session of Ubuntu. In this Chinese Ubuntu session I am trying to run an English instance of Firefox by prepending "env LANG=en_US.UTF-8". Note how Firefox comes up in Chinese instead of English.

[Gunnar Hjalmarsson (gunnarhj)]

@dreamon
You are probably right when assuming that your observations are related to the fixes of this bug.

If your regional formats setting (second tab in Language Support) is something else but "Chinese (China)", your choosing "Chinese (China)" as the display language makes gdm set the LC_MESSAGES variable to "zh_CN.UTF-8". Since LC_MESSAGES overrides LANG, setting LANG before launching an app is not sufficient. Try this instead:

env LC_MESSAGES=en_US.UTF-8 firefox

[dreamon (dreamon)]

Gunnar, thank you for your quick reply. Unfortunately, this didn't make a difference. Firefox still comes up in Chinese. This is on a fresh Natty install, by the way (had to reinstall since my last post for a different reason).

Suppose that means the bug is not fixed then?

Any other suggestions? Any additional test you'd like me to run?

[Gunnar Hjalmarsson (gunnarhj)]

It makes a difference to me.

Since you said "an English instance of Firefox", maybe I should point out that you can't have a Chinese and an English FF instance simultaneously. In other words, Firefox must not be running when you execute the command I suggested.

[dreamon (dreamon)]

Yes, of course. Firefox wasn't running when I executed the env LC_MESSAGES and env LANG commands. I also tried running the command with other programmes, such as gedit, but always got the same result: contrary to my expectation, the programme would not open up in the language specified, but in the language I picked at login. This didn't happen on Lucid, and, as I pointed out above, stopped happening on Maverick.

Do other users also experience this problem? Shall we track this in a separate bug report?

[Gunnar Hjalmarsson (gunnarhj)]

Strange. Firefox should be started in English if you set LC_MESSAGES as I suggested. Typo, maybe? ;-)

gedit is another thing. Unlike Firefox gedit recognizes the LANGUAGE env. variable, which overrides both LANG and LC_MESSAGES, so to make gedit be started in English you would need:

env LANGUAGE=en gedit

Yes, if there is a need to track it down, please submit a separate bug.

[dreamon (dreamon)]

Thanks again for your suggestions, Gunnar. I copied your commands directly from here to the terminal, so it is not a typo.

Strangely, setting env LANG with firefox or thunderbird actually worked on the last three Ubuntu versions, even though it may be incorrect to run it this way. I was able to run an English instance of Thunderbird, for example, by using env LANG.

I have opened a separate bug report here: https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/792735 and supplied more information to track down the problem. Any suggestions would be welcome!

[Juan Simón (simonbcn)]

This still fails on Natty 64 bits. I have defined:

$ cat /etc/default/locale
LANG="es_ES.UTF-8"
LC_ALL=
LC_MESSAGES=POSIX
LC_COLLATE=C

But when I enter in Gnome:
$ locale
LANG=es_ES.UTF-8
LANGUAGE=es_ES:en
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE=C
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=

Only it respects LC_COLLATE but not LC_MESSAGES.

[Gunnar Hjalmarsson (gunnarhj)]

Hi Simon,

/etc/default/locale contains system wide settings that basically affect the startup and the login screen. Once logged in, your personal language settings override the system wide ditto.

Please select English as your user language, either from the language chooser at the bottom of the login screen or from Language Support.

I recommend that you click the "Help" button on the Language Support window for a description of how things are intended to work.

[hardik (hardik-soni)]

see this also
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/820333