13.10 to 14.04 upgrade: UnicodeEncodeError: 'ascii' codec can't encode character u'\xab' in position 13: ordinal not in range(128) Logged from file DistUpgradeCache.py, line 661 in apt.log

Bug #1310053 reported by Patrizio Bruno on 2014-04-19
94
This bug affects 16 people
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
High
Michael Vogt
Trusty
High
Unassigned

Bug Description

TEST CASE:
- install raring and ubuntu-release-upgrader-qt
- use a language like de, it or fr
- run do-release-upgrader --frontend DistUpgradeViewText
- verify that you see a error message in apt.log as described in the bug (unfortunately not everyone can reproduce this issue, I reproduced it by editing DistUpgradeCache.py to always throw a blacklisted package expection)
- if the error does not show up, canncel the upgrade, run "vi /tmp/*/removal_blacklist.cfg" and add a package from the removal list that was displayed during the upgrade and run /tmp/*/trusty

- run the upgrade using "do-release-upgrade --proposed --frontend DistUpgradeViewText"
- verify that there is no exception in apt.log

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: ubuntu-release-upgrader-core 1:0.205.4
ProcVersionSignature: Ubuntu 3.11.0-15.23-generic 3.11.10
Uname: Linux 3.11.0-15-generic i686
ApportVersion: 2.12.5-0ubuntu2.2
Architecture: i386
CrashDB: ubuntu
Date: Sat Apr 19 19:50:56 2014
MarkForUpload: True
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
UpgradeStatus: Upgraded to saucy on 2014-04-19 (0 days ago)

Jean-Baptiste Lallement (jibel) wrote :

From apt.log:

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 851, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 724, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
UnicodeEncodeError: 'ascii' codec can't encode character u'\xab' in position 13: ordinal not in range(128)
Logged from file DistUpgradeCache.py, line 661

summary: - trying to update to 14.04
+ trying to update to 14.04: UnicodeEncodeError: 'ascii' codec can't
+ encode character u'\xab' in position 13: ordinal not in range(128)
+ Logged from file DistUpgradeCache.py, line 661 in apt.log
Changed in ubuntu-release-upgrader (Ubuntu):
importance: Undecided → High
status: New → Confirmed
summary: - trying to update to 14.04: UnicodeEncodeError: 'ascii' codec can't
- encode character u'\xab' in position 13: ordinal not in range(128)
- Logged from file DistUpgradeCache.py, line 661 in apt.log
+ 13.10 to 14.04 upgrade: UnicodeEncodeError: 'ascii' codec can't encode
+ character u'\xab' in position 13: ordinal not in range(128) Logged from
+ file DistUpgradeCache.py, line 661 in apt.log
Brian Murray (brian-murray) wrote :

This is the problematic line:

            logging.error("Dist-upgrade failed: '%s'", e)

Maybe, the comma is causing the issue.

Changed in ubuntu-release-upgrader (Ubuntu):
milestone: none → trusty-updates
Brian Murray (brian-murray) wrote :

Some experiencing this bug could test the the theory by editing the file DistUpgradeCache.py in /tmp/ubuntu-release-upgrader-TMPNAME and then cd'ing into that directory and running sudo ./trusty and seeing if they get the same traceback.

Well I can confirm that line 661 was the one that tried to log error, but changing
logging.error("Dist-upgrade failed: '%s'", e)
to
logging.error("Dist-upgrade failed: '%s'", unicode(e))

only results in correctly printing 'Dist-upgrade' failed error to log file:

2014-04-22 20:06:46,539 DEBUG The package 'update-manager-kde' is marked for removal but it's in the removal blacklist
2014-04-22 20:06:49,343 ERROR Dist-upgrade failed: 'Pakiet "update-manager-kde" jest oznaczony do usunięcia, lecz znajduje się na liście pakietów, których nie należy usuwać.'
2014-04-22 20:06:49,345 DEBUG abort called

and does NOT fix the root cause. (Meaning that this change only adds one line to logfile).

I might add that u'xab' appears to be character « according to http://es.wikipedia.org/wiki/ISO_8859-1 (and my test with python2.7).

Changed in ubuntu-release-upgrader (Ubuntu Trusty):
status: New → In Progress
importance: Undecided → High
milestone: none → trusty-updates
Changed in ubuntu-release-upgrader (Ubuntu):
milestone: trusty-updates → none
status: Confirmed → Triaged

Thanks for uploading the fix for this bug report to -proposed. However, when reviewing the package in -proposed and the details of this bug report I noticed that the bug description is missing information required for the SRU process. You can find full details at http://wiki.ubuntu.com/StableReleaseUpdates#Procedure but essentially this bug is missing some of the following: a statement of impact, a test case and details regarding the regression potential. Thanks in advance!

Changed in ubuntu-release-upgrader (Ubuntu):
assignee: nobody → Michael Vogt (mvo)
Michael Vogt (mvo) on 2014-04-29
description: updated
description: updated

Hello Patrizio, or anyone else affected,

Accepted ubuntu-release-upgrader into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:0.220.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in ubuntu-release-upgrader (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed

Sorry for delay, but I cannot install 1:0.220.3 . I enabled proposed, run
apitude update
but all I see in aptitude versions ubuntu-release-upgrader
is
1:0.205.7 saucy-proposed 500
Is 64bit package is in repo already? I selected Ubuntu main repo instead of local mirror but it did not help.

Brian Murray (brian-murray) wrote :

To use the upgrader from Trusty proposed you need to use the following command:

"do-release-upgrade --proposed --frontend DistUpgradeViewText"

Notice the --proposed switch. This is because the release upgrade process downloads a new version of itself from the release you are upgrading to.

Ok I tested it on pl locale. Looks like in newer version it looks better, because I got

DEBUG The package 'update-manager-kde' is marked for removal but it's in the removal blacklist

line in main.log (which previoulsy was not there), but unfortunetlly my proposed fix for DistUpgradeCache was invalid
since error was still shown in apt.log :
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0119' in position 51: ordinal not in range(128)
Logged from file DistUpgradeCache.py, line 661

(\u0119 i ę character which probably appeared in polish version of blacklist line:
Dist-upgrade failed: 'Pakiet "update-manager-kde" jest oznaczony do usunięcia, lecz znajduje się na liście pakietów, których nie należy usuwać.'
I do not know why it try to encode this line when it shown english version in main.log)

I attach patch that fixes apt.log issue for me.

tags: added: verification-failed
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:14.10.2

---------------
ubuntu-release-upgrader (1:14.10.2) utopic; urgency=low

  * debian/test/control:
    - add missing pep8 test dependency
  * tests/test_xorg_fix_intrepid.py:
    -
 -- Michael Vogt <email address hidden> Sat, 03 May 2014 08:37:51 +0200

Changed in ubuntu-release-upgrader (Ubuntu):
status: Triaged → Fix Released
Woko (wkoehn) wrote :

I still get the mentioned error-messages.
What has to be done to benefit from the fix ?

Wolfram Köhn (wk-i) wrote :

I finaly succeeded upgrading to 14.04.
The simple solution was to deinstall 'update-manager-kde' manually before restarting the upgrade.

Changed in ubuntu-release-upgrader (Ubuntu Trusty):
status: Fix Committed → In Progress
Brian Murray (brian-murray) wrote :

I've gone ahead and recreated this bug following the description by adding foomatic-filters to the blacklist. I received the Traceback indicated with the ubuntu-release-upgrader version 0.220.2, but not with 0.220.3. I was using French for my test. Additionally, I did see unicode characters in my /var/log/dist-upgrade/main.log file.

The message included "ERROR Dist-upgrade failed: 'Le paquet << foomatic-fitlers >> est marqué". The Dist-upgrade failure should not appear in /var/log/dist-upgrade/apt.log so I'm not certain what Sebastian was seeing, but I'm going to mark this as verification done.

Changed in ubuntu-release-upgrader (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-done
removed: verification-failed
Steve Langasek (vorlon) wrote :

Hello Patrizio, or anyone else affected,

Accepted ubuntu-release-upgrader into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:0.220.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: removed: verification-done
tags: added: verification-needed
tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:0.220.4

---------------
ubuntu-release-upgrader (1:0.220.4) trusty-proposed; urgency=low

  * DistUpgrade/DistUpgradeCache.py:
    - fix crash in free space check for certain locales (LP: #1172740)
    - removal of essential packages that are not in the main arch
      is ok (LP: #1307904)
    - fix str/unicode crash in certain locales when a package is
      found in the removal blacklist (LP: #1310053)
  * DistUpgrade/DistUpgradeViewGtk3.py:
   - make details dialog resize when details are expanded/collapsed
     (LP: #1072460)
    - load webkit only when needed
  * DistUpgrade/DistUpgradeViewText.py:
    - fix py3 crash in text mode
  * DistUpgrade/DistUpgradeMain.py:
    - log python version
  * po/{eo,id,ug}.po:
    - fix translations of check-new-release-gtk which cause a crash
      (LP: #1311396)
  * data/demoted.cfg:
    - autmatic refresh of the demoted packages
  * po/*.po:
    - automatic refresh from launchpad
 -- Michael Vogt <email address hidden> Thu, 24 Apr 2014 14:57:39 +0200

Changed in ubuntu-release-upgrader (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers