do-release-upgrade crashes with message : "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 337: invalid start byte"

[Philippe Schelte]

for dist upgrade from Ubuntu 14.04 LTS to the next "normal" release, when I make a do-release-upgrade -d, I get this error :
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position
337: invalid start byte
I have cleaned all the files in /etc/apt/sources.list.d/
but no change in order to remove all unencoded characters like accent etc...

# lsb_release -rd
Description: Ubuntu 14.04.2 LTS
Release: 14.04

[David Peall (dkpeall)]

Please report this as a bug and include the files
/var/log/dist-upgrade/main.log and /var/log/dist-upgrade/apt.log in
your report. The upgrade has aborted.
Your original sources.list was saved in
/etc/apt/sources.list.distUpgrade.

Traceback (most recent call last):

File "/tmp/ubuntu-release-upgrader-zbnoluzb/utopic", line 8, in
<module>
sys.exit(main())

File
"/tmp/ubuntu-release-upgrader-zbnoluzb/DistUpgrade/DistUpgradeMain.py",
line 239, in main
if app.run():

File
"/tmp/ubuntu-release-upgrader-zbnoluzb/DistUpgrade/DistUpgradeController.py",
line 1822, in run
return self.fullUpgrade()

File
"/tmp/ubuntu-release-upgrader-zbnoluzb/DistUpgrade/DistUpgradeController.py",
line 1707, in fullUpgrade
if not self.doPostInitialUpdate():

File
"/tmp/ubuntu-release-upgrader-zbnoluzb/DistUpgrade/DistUpgradeController.py",
line 893, in doPostInitialUpdate
self.tasks = self.cache.installedTasks

File
"/tmp/ubuntu-release-upgrader-zbnoluzb/DistUpgrade/DistUpgradeCache.py",
line 778, in installedTasks
for line in pkg._pcache._records.record.split("\n"):

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position
403: invalid start byte
=== Command terminated with exit status 1 (Fri May 29 09:35:36 2015) ===

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robert Krambovitis (qobert)]

I just encountered this issue when trying to update a trusty server to xenial.
Here's how we "resolved" it.

After do-release-update exited with a similar to the above message, there is a temporary ubuntu-release-upgrader-{random} left over in /tmp.
Enter that directory i.e. cd /tmp/ubuntu-release-upgrader-asd123
edit DistUpgradeCache.py
Head down to def installedTasks(self)
after the line that reads: "for pkg in self:"
add logging.debug("%s \n" % pkg)
You need to make sure this is indented with spaces to the same level as the next line.

save / exit, then run (assuming you are upgrading to xenial)
./xenial

When it crashes this time, you can view /var/log/dist-upgrade/main.log and see which was the last package processed that caused the crash. Remove this package, then proceed with do-release-upgrade as normal. In our case it was a very old wowzaServer package.

This is obviously not a fix, but it got the job done :)
Hope this helps.