Locale issues with beta-1/2 cloud-images

Bug #859814 reported by James Page
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Undecided
Scott Moser
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

Default locale of image appears to be en_US.UTF-8 - however when I logon from a en_GB.UTF-8 client via SSH I get the following errors - springs up in a few places while doing things on the server.

This was from a local openstack test instance; however I get the same error in ec2 instances with the latest daily snapshot.

ubuntu@server-3:~$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=en_GB.UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=en_GB.UTF-8
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES=en_GB.UTF-8
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Related Bugs:
 * Bug 920601: cloud-images don't accept LANG settings
 * Bug 969462: postgresql-9.1 fails to start after install if invalid locale is set

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: cloud-init 0.6.1-0ubuntu15
ProcVersionSignature: User Name 3.0.0-9.15-virtual 3.0.3
Uname: Linux 3.0.0-9-virtual x86_64
Architecture: amd64
Date: Mon Sep 26 17:09:24 2011
Ec2AMI: ami-00000002
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: m1.small
Ec2Kernel: aki-00000001
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 LC_CTYPE=en_GB.UTF-8
 LC_COLLATE=en_GB.UTF-8
 LANG=en_US.UTF-8
 LC_MESSAGES=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Revision history for this message
James Page (james-page) wrote :
description: updated
summary: - Locale issues with beta-2 cloud-images
+ Locale issues with beta-1/2 cloud-images
Dave Walker (davewalker)
tags: added: server-o-rs
Revision history for this message
Scott Moser (smoser) wrote :

I can confirm this, and also confirm it on 11.04. I suspect it is the case with even 10.04.

$ env LC_ALL=en_GB.UTF-8 ssh 10.55.60.59 'cat /etc/cloud/build.info; echo ===; locale'
bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.UTF-8)
build_name: server
serial: 20110426
===
locale: Cannot set LC_CTYPE to default localeLANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=en_GB.UTF-8
: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

I also suspect this is the case with *any* server that does not have locale X installed if you ssh in with a locale set locally.

Changed in cloud-init (Ubuntu):
importance: Undecided → Low
status: New → Confirmed
Revision history for this message
Dave Walker (davewalker) wrote :

ubuntu@server-1251:~$ cat /etc/cloud/build.info
build_name: server
serial: 20110921.1

ubuntu@server-1251:~$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Changed in cloud-init (Ubuntu):
status: Confirmed → Incomplete
Dave Walker (davewalker)
Changed in cloud-init (Ubuntu):
milestone: none → ubuntu-11.10
Dave Walker (davewalker)
Changed in cloud-init (Ubuntu):
milestone: ubuntu-11.10 → none
tags: removed: server-o-rs
Revision history for this message
Scott Moser (smoser) wrote :

There was some discussion on this, but i believe the only real way to fix it is to add locales to the image.

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for cloud-init (Ubuntu) because there has been no activity for 60 days.]

Changed in cloud-init (Ubuntu):
status: Incomplete → Expired
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
assignee: nobody → Scott Moser (smoser)
importance: Low → Medium
status: Expired → In Progress
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
assignee: Scott Moser (smoser) → Ben Howard (utlemming)
Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

So a couple of findings here.

We are doing locales wrong: generating the locales is only part of the process of localizations. So while Cloud-Init generates the locales, the images are far from localized. In order to have a localized image, the base language pack (i.e. language-pack-{en,fr,de,...} needs to be installed otherwise the base language remains in English. The default "en_US.UTF8" locale is incomplete in the images since the language pack is not installed.

Therefore, I think that we should do the following:

1. The images should have the "language-pack-en" installed. Any English speaker who is exporting variable via SSH will automatically get localization as long as the localization is standard. Since Ubuntu's language of collaberation is English, this should fix the vast majority of users encountering localization problems, unless they are exporting non-standard chacter sets (i.e. exporting en_GB instead of en_GB.UTF8).

2. Cloud-init will should support the installation of language packs when it parses "locale: <locale>" during the cloud-config section.

3. Cloud-init should have a /etc/profile.d message that tells people if their locale settings are wrong and how to fix it.

Scott Moser (smoser)
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

Some more data here, the following results in postgresql not started on installation:

$ LC_ALL=en_GB.UTF-8 ssh ubuntu@$HOSTNAME
$ sudo apt-get install postgresql-9.1
...
Adding user postgres to group ssl-cert
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Setting up postgresql-9.1 (9.1.3-2) ...
Error: The locale requested by the environment is invalid.
Error: could not create default cluster. Please create it manually with

  pg_createcluster 9.1 main --start

or a similar command (see 'man pg_createcluster').

If you run 'apt-get install language-pack-en' prior to running the above, it will work.

It will still fail, though if you do:
 $ LC_ALL=fr_FR.UTF-8 ssh ....
 $ sudo apt-get install postgresql-9.1
(which would be fixed by 'apt-get install language-pack-fr').

Scott Moser (smoser)
description: updated
Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Added "language-pack-en" to all Precise builds.

Revision history for this message
Scott Moser (smoser) wrote :

Ben, the right way to do this is for us to add 'language-pack-en' to the cloud-image seed at http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/ubuntu.precise/files .

Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Scott Moser (smoser) wrote :

i've added language-pack-en to the cloud-images seed. So we can back out the automated-ec2-builds changes.

Revision history for this message
Scott Moser (smoser) wrote :

Ug. whenever we think we're going down the right track, we find more information. But I guess, better to know it now then later.
http://irclogs.ubuntu.com/2012/04/03/%23ubuntu-devel.html#t00:19 is a conversation between cjwatson, myself and infinity.

There, I find out
 a. it is perfectly valid and supported to run 'localegen' for locales where no corresponding language-pack-xx is installed
 b. an easy way to check "is this locale currently valid" is to do:
    import locale
    try:
       locale.setlocale(locale.LC_ALL, '') # or a specific locale as the second argument
    except locale.Error

That wont' involve apt (and thus will be quicker).

Because of 'a', cloud-init is not invalid in doing its 'locale-gen' on possibly un-installed locales, so I think i'd just leave that as it is for now.

And i think we can just handle package installation via informing the user on login. This means really that the more difficult to get right portion of this change (the package installation) can just be skipped.

What do you think, Ben?

Revision history for this message
Scott Moser (smoser) wrote :

OK.
  So in talking with Ben, we've come up with the following plan:
 * remove the language-pack-en from the cloud-image seed (i just did this and pushed to lp:~ubuntu-core-dev/ubuntu-seeds/ubuntu.precise/ revision 2022).
 * leave cloud-init's locale generation code alone for 12.04
 * add the profile.d warning based on lp:~utlemming/cloud-init/cloud-init.profile.d and show that the first time the user logs in with invalid locale.

Revision history for this message
Scott Moser (smoser) wrote :

Fix-committed in revision 553.

Changed in cloud-init:
assignee: nobody → Scott Moser (smoser)
status: New → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote :

Ugh.
I put a bad bug number in the changelog, but
cloud-init (0.6.3~bzr554-0ubuntu1) precise; urgency=low
    * Provide user-friendly message when a user ssh's in with an invalid
      locale (LP: #960547)

Marking fix-released in ubuntu

Changed in cloud-init (Ubuntu):
status: In Progress → Fix Released
Scott Moser (smoser)
Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.