Invalid encoding returned for ca@valencia locale

[David Planella]

While upgrading from 13.04 to 13.10 using locale ca@valencia, the process fails with the following traceback:

Traceback (most recent call last):

File "/tmp/ubuntu-release-upgrader-vh1jgw/saucy", line 10, in
<module>
sys.exit(main())

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeMain.py",
line 243, in main
if app.run():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1791, in run
return self.fullUpgrade()

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1744, in fullUpgrade
if not self.askDistUpgrade():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 996, in askDistUpgrade
self.cache.required_download)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeViewText.py",
line 165, in confirmChanges
downloadSize, actions)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeView.py",
line 387, in confirmChanges
downloadSizeStr = downloadSizeStr.decode(ENCODING)

LookupError: unknown encoding: utf_8_valencia

Looking at it in more detail, this comes from the following code in DistUpgrade/DistUpgradeView.py [1]

try:
    locale.setlocale(locale.LC_ALL, "")
    (code, ENCODING) = locale.getdefaultlocale()
except:
    logging.exception("getting the encoding failed")
    ENCODING = "utf-8" #pyflakes

if not ENCODING:
    ENCODING = "utf-8"

And digging deeper, it seems getdefaultlocale does not return the correct encoding when the locale is set to ca_ES

>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'LC_CTYPE=ca_ES.UTF-8@valencia;LC_NUMERIC=ca_ES.UTF-8;LC_TIME=ca_ES.UTF-8;LC_COLLATE=ca_ES.UTF-8@valencia;LC_MONETARY=ca_ES.UTF-8;LC_MESSAGES=ca_ES.UTF-8@valencia;LC_PAPER=ca_ES.UTF-8;LC_NAME=ca_ES.UTF-8;LC_ADDRESS=ca_ES.UTF-8;LC_TELEPHONE=ca_ES.UTF-8;LC_MEASUREMENT=ca_ES.UTF-8;LC_IDENTIFICATION=ca_ES.UTF-8'
>>> locale.getdefaultlocale()
('ca_ES', 'utf_8_valencia')

Instead of 'utf_8_valencia', here I'd expect 'UTF-8' to be returned. The returned locale is also wrong: it should be ca_ES@valencia.

While this needs to be fixed upstream, it might be worth adding a workaround (and a check for a valid encoding) in ubuntu-release-upgrader in the meantime.

This will also affect other @ locales, such as sr@Latin (Serbian with Latin encoding).

[1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/view/head:/DistUpgrade/DistUpgradeView.py#L41

[Xavi Ivars (xavi-ivars)]

This bug is still open, and it also affects updating from 13.10 to 14.04, even if the triaged bug in python was fixed in december, four months ago.

[David Planella (dpm)]

I think the issue here is that while the fix for the upstream bug was committed [1] on the 19th Dec 2013, there hasn't been an upstream 2.7 point release that includes it, and thus that it can be packaged for Ubuntu.

If I'm not mistaken, the latest upstream point release (2.7.6) was on the 10th of November [2]. While it hasn't yet made it to Ubuntu (we've got 2.7.5, and 2.7.6 might not qualify for a Stable Release Upload), the main point is that the latest upstream release was made about one month before the fix was committed, thus there is no release from the 2.7.x series that includes it.

I think there is not an easy fix for Ubuntu 13.10 (backporting the fix might will most probably not qualify as SRU, and porting ubuntu-release-upgrader to Python 3.3, which might have the fix in Ubuntu 13.10 is probably not feasible for a stable release).

So in summary, this will probably be a "Won't fix" for Ubuntu 13.10. Not sure what to do with 14.04, as it might have exactly the same issue and we'll have the same constraints now that it has been released.

[1] http://hg.python.org/cpython/rev/3d805bee06e2
[2] https://www.python.org/download/releases/2.7.6
[3] https://launchpad.net/ubuntu/saucy/+source/python2.7/+changelog

[Chris Guiver (guiverc)]

Thank you for reporting this bug to Ubuntu.

Ubuntu 13.10 (saucy) reached end-of-life on July 17, 2014.
Ubuntu 13.04 (raring) reached end-of-life on January 27, 2014.

See this document for currently supported Ubuntu releases:
https://wiki.ubuntu.com/Releases

We appreciate that this bug may be old and you might not be interested in discussing it any more. But if you are then please upgrade to the latest Ubuntu version and re-test. If you then find the bug is still present in the newer Ubuntu version, please add a comment here telling us which new version it is in.