do-release-upgrade fails

[Rob Owens]

Running a completely up to date Ubuntu 20.04, I run:

sudo do-release-upgrade

and get the following output:

<snipped the normal stuff above>
Building dependency tree
Reading state information... Done

Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/jammy", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeMain.py", line 241, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 2042, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 1873, in fullUpgrade
    if not self.doPostInitialUpdate():
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 906, in doPostInitialUpdate
    self.quirks.run("PostInitialUpdate")
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 99, in run
    func()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 122, in jammyPostInitialUpdate
    self._test_and_fail_on_pam_tally()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 395, in _test_and_fail_on_pam_tally
    content = f.read()
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xec in position 8200: invalid continuation byte
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/problem_report.py", line 477, in add_to_existing
    self.write(f)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 430, in write
    block = f.read(1048576)
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Original exception was:
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/jammy", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeMain.py", line 241, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 2042, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 1873, in fullUpgrade
    if not self.doPostInitialUpdate():
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeController.py", line 906, in doPostInitialUpdate
    self.quirks.run("PostInitialUpdate")
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 99, in run
    func()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 122, in jammyPostInitialUpdate
    self._test_and_fail_on_pam_tally()
  File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 395, in _test_and_fail_on_pam_tally
    content = f.read()
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xec in position 8200: invalid continuation byte
Press x to destroy or r to resurrect window t Aug 13 10:07:11 2022) ===

$ lsb_release -rd
Description: Ubuntu 20.04.4 LTS
Release: 20.04

$ apt-cache policy python3.8
python3.8:
  Installed: 3.8.10-0ubuntu1~20.04.5
  Candidate: 3.8.10-0ubuntu1~20.04.5
  Version table:
 *** 3.8.10-0ubuntu1~20.04.5 500
        500 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.8.2-1ubuntu1 500
        500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages

The only thing unusual about my installation is that I'm on a zfs root.

[Rob Owens (hick518)]

On Feb 26 I opened this bug, which has not had any activity since I opened it:
https://bugs.launchpad.net/ubuntu/+source/linux-hwe-5.13/+bug/1962373

I was hoping that maybe an upgrade to 22.04 would make the above irrelevant, but now I'm not able to 'do-release-upgrade' due to this bug. Can anybody give me some assurance that this bug won't end up like the last bug I filed?

I can be patient if this bug is going to be worked on, but if it's not then I should probably start making plans to move to another distro that will work with my hardware.

[Rob Owens (hick518)]

I managed to figure this out. I'm gonna say the bug here is poor logging.

In the output I posted, in the "Original exception was" section, the original error is:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xec in position 8200: invalid continuation byte

Go back up a couple lines and you'll see this:

File "/tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/DistUpgradeQuirks.py", line 395, in _test_and_fail_on_pam_tally

This is the upgrader script that was running when the error was thrown. Edit that file (as root) and go to line 395 and see that you're in a "for" loop:

        for f in os.listdir('/etc/pam.d'):
            if f in ('common-account', 'common-auth', 'common-password',
                     'common-session', 'common-session-noninteractive'):
                # managed by pam-auth-updates, and any references to
                # pam_tally* will be handled by libpam-modules on upgrade
                # without breaking
                continue
            with open(os.path.join('/etc/pam.d', f)) as f:
                content = f.read()
                if re.search('^[^#]*pam_tally', content, re.MULTILINE): <--- line 395
            <snipped>

The script is looping through files in /etc/pam.d. You can go look there for the trouble, or you can edit the script to find which file is being processed when the error is thrown. Like this:

        for f in os.listdir('/etc/pam.d'):
            print ("Rob says: " + f)
            <snipped>

Now run the upgrader again from this temporary directory, which includes the edits made above. Like this:

sudo /tmp/ubuntu-release-upgrader-l3yamuwr/DistUpgrade/jammy

We run the "jammy" script because that's what we're trying to upgrade to. If in doubt, do 'ls -l' on that directory and look for an executable script that is named after an ubuntu release.

After you run the above command, you'll still get an error but if you search the output for "Rob says", you'll see the list of files that are being processed in the "for" loop. The last one in the list before the error is the likely culprit. In my case it was:

.common-auth.swp

This is a vi swap file. Several months ago I apparently edited common-auth and didn't save it before the computer was shut down. I deleted that file and then ran:

sudo do-release-upgrade

and the upgrade is now running.

So the problem was that the upgrader tried to process a file but it could not (because it was not an ascii file). The upgrader should have, at the very least, output the name of the problematic file.

[Andrij G. (gera-b)]

thanks, the fix works.

The part with logging inserted should look like this:

 with open(os.path.join('/etc/pam.d', f)) as f:
     for f in os.listdir('/etc/pam.d'):
            print ("Rob says: " + f)
     content = f.read()

and yes it was also an *.swp file in my case.

Thanks for your research

[ferrouswheel (joel-pitt)]

Thanks - this fix worked for me. In this case it was a vim *.un~ undo file.