Invalid encoding returned for ca@valencia locale

Bug #1243065 reported by David Planella on 2013-10-22
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Python
Confirmed
Unknown
ubuntu-release-upgrader (Ubuntu)
High
howard goddard

Bug Description

While upgrading from 13.04 to 13.10 using locale ca@valencia, the process fails with the following traceback:

Traceback (most recent call last):

File "/tmp/ubuntu-release-upgrader-vh1jgw/saucy", line 10, in
<module>
sys.exit(main())

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeMain.py",
line 243, in main
if app.run():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1791, in run
return self.fullUpgrade()

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1744, in fullUpgrade
if not self.askDistUpgrade():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 996, in askDistUpgrade
self.cache.required_download)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeViewText.py",
line 165, in confirmChanges
downloadSize, actions)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeView.py",
line 387, in confirmChanges
downloadSizeStr = downloadSizeStr.decode(ENCODING)

LookupError: unknown encoding: utf_8_valencia

Looking at it in more detail, this comes from the following code in DistUpgrade/DistUpgradeView.py [1]

try:
    locale.setlocale(locale.LC_ALL, "")
    (code, ENCODING) = locale.getdefaultlocale()
except:
    logging.exception("getting the encoding failed")
    ENCODING = "utf-8" #pyflakes

if not ENCODING:
    ENCODING = "utf-8"

And digging deeper, it seems getdefaultlocale does not return the correct encoding when the locale is set to ca_ES

>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'LC_CTYPE=ca_ES.UTF-8@valencia;LC_NUMERIC=ca_ES.UTF-8;LC_TIME=ca_ES.UTF-8;LC_COLLATE=ca_ES.UTF-8@valencia;LC_MONETARY=ca_ES.UTF-8;LC_MESSAGES=ca_ES.UTF-8@valencia;LC_PAPER=ca_ES.UTF-8;LC_NAME=ca_ES.UTF-8;LC_ADDRESS=ca_ES.UTF-8;LC_TELEPHONE=ca_ES.UTF-8;LC_MEASUREMENT=ca_ES.UTF-8;LC_IDENTIFICATION=ca_ES.UTF-8'
>>> locale.getdefaultlocale()
('ca_ES', 'utf_8_valencia')

Instead of 'utf_8_valencia', here I'd expect 'UTF-8' to be returned. The returned locale is also wrong: it should be ca_ES@valencia.

While this needs to be fixed upstream, it might be worth adding a workaround (and a check for a valid encoding) in ubuntu-release-upgrader in the meantime.

This will also affect other @ locales, such as sr@Latin (Serbian with Latin encoding).

[1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/view/head:/DistUpgrade/DistUpgradeView.py#L41

Changed in python:
status: Unknown → New
Changed in python:
status: New → Confirmed
Changed in ubuntu-release-upgrader (Ubuntu):
status: New → Triaged
importance: Undecided → High
Xavi Ivars (xavi-ivars) wrote :

This bug is still open, and it also affects updating from 13.10 to 14.04, even if the triaged bug in python was fixed in december, four months ago.

David Planella (dpm) wrote :

I think the issue here is that while the fix for the upstream bug was committed [1] on the 19th Dec 2013, there hasn't been an upstream 2.7 point release that includes it, and thus that it can be packaged for Ubuntu.

If I'm not mistaken, the latest upstream point release (2.7.6) was on the 10th of November [2]. While it hasn't yet made it to Ubuntu (we've got 2.7.5, and 2.7.6 might not qualify for a Stable Release Upload), the main point is that the latest upstream release was made about one month before the fix was committed, thus there is no release from the 2.7.x series that includes it.

I think there is not an easy fix for Ubuntu 13.10 (backporting the fix might will most probably not qualify as SRU, and porting ubuntu-release-upgrader to Python 3.3, which might have the fix in Ubuntu 13.10 is probably not feasible for a stable release).

So in summary, this will probably be a "Won't fix" for Ubuntu 13.10. Not sure what to do with 14.04, as it might have exactly the same issue and we'll have the same constraints now that it has been released.

[1] http://hg.python.org/cpython/rev/3d805bee06e2
[2] https://www.python.org/download/releases/2.7.6
[3] https://launchpad.net/ubuntu/saucy/+source/python2.7/+changelog

Changed in ubuntu-release-upgrader (Ubuntu):
assignee: nobody → howard goddard (hgoddard2)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.