18.04 to 20.04 upgrade failed until -dbg packages removed

[Zach Boyd]

I've uploaded the logs. Based on the printout

"Could not calculate the upgrade

An unresolvable problem occurred while calculating the upgrade.

If none of this applies, then please report this bug using the
command 'ubuntu-bug ubuntu-release-upgrader-core' in a terminal. If
you want to investigate this yourself the log files in
'/var/log/dist-upgrade' will contain details about the upgrade.
Specifically, look at 'main.log' and 'apt.log'."

I am filing this. Note that the message seems to expect to be able to list some reasons for the failure but doesn't have any. In an earlier run, it asked me to run ppa-purge, after which the message change to give no suggested causes. (I do suspect that I missed something at that step since there are 206 broken packages in the apt.log file still. Maybe I accidentally removed some ppas rather than doing ppa-purge? I am unable to tell from the apt.log file which if any of the Broken packages is blocking the upgrade.)

In main.log, it just says "ERROR Dist-upgrade failed: 'E:Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.'" I checked, and there are no held packages.

Really struggling to find a lead to resolve this.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: ubuntu-release-upgrader-core 1:18.04.42
ProcVersionSignature: Ubuntu 4.15.0-135.139-generic 4.15.18
Uname: Linux 4.15.0-135-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.23
Architecture: amd64
CrashDB: ubuntu
CurrentDesktop: ubuntu:GNOME
Date: Fri Feb 5 13:35:06 2021
InstallationDate: Installed on 2019-08-13 (541 days ago)
InstallationMedia: Ubuntu 18.04.1 LTS "Bionic Beaver" - Release amd64 (20180725)
PackageArchitecture: all
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: ubuntu-release-upgrader
UpgradeStatus: Upgraded to bionic on 2021-02-05 (0 days ago)

[Jouni Mettala (jouni-mettala)]

Reason could be python-dbg or python-dev packages during upgrade.

Could you remove following packages before upgrade?
libpython2-dbg python2-dbg libpython2-dev python2-dev python-dev libpython-all-dev python-dev

[Zach Boyd (zboyd2)]

Thanks! Just tried removing them with apt purge. Some could not be found. do-release upgrade is still giving (to me) similar output as before.

[Jouni Mettala (jouni-mettala)]

there is still pkgProblemResolver with broken count: 31

These debugging packages should be safe to remove:
bluez-dbg 5.48-0ubuntu3.4 0
libbluetooth3-dbg 5.48-0ubuntu3.4 1
libc6-dbg 2.27-3ubuntu1.4 0
libpython-dbg 2.7.15~rc1-1 1
libpython2.7-dbg 2.7.17-1~18.04ubuntu1.2 1
libpython3-dbg 3.6.7-1~18.04 1
libpython3.6-dbg 3.6.9-1~18.04ubuntu1.3 1
linux-image-4.15.18-custom-dbg 4.15.18-custom-1 0
python-dbg 2.7.15~rc1-1 0
python-numpy-dbg 1:1.13.3-2ubuntu1 0
python2.7-dbg 2.7.17-1~18.04ubuntu1.2 1
python3-dbg 3.6.7-1~18.04 0
python3-numpy-dbg 1:1.13.3-2ubuntu1 0
python3.6-dbg 3.6.9-1~18.04ubuntu1.3 1

Could you attach apt.log after removing these -dbg packages?
It should at least reduce broken count a lot.
Perhaps file needs to be renamed before attaching.

If upgrade still fails you could try removing obsolete packages and/or python-dev packages:

DEBUG Obsolete: linux-headers-4.15.18-custom linux-image-4.15.18-custom linux-image-4.15.18-custom-dbg python3-graph-tool rstudio slack-desktop texlive-local wolframscript zoom

python-all-dev 2.7.15~rc1-1 1
python-dev 2.7.15~rc1-1 0
python2.7-dev 2.7.17-1~18.04ubuntu1.2 1
python3-dev 3.6.7-1~18.04 0
python3.6-dev 3.6.9-1~18.04ubuntu1.3 1
python3.8-dev 3.8.0-3~18.04 0
libpython-all-dev 2.7.15~rc1-1 1
libpython-dev 2.7.15~rc1-1 1
libpython2.7-dev 2.7.17-1~18.04ubuntu1.2 1
libpython3-dev 3.6.7-1~18.04 1
libpython3.6-dev 3.6.9-1~18.04ubuntu1.3 1
libpython3.8-dev 3.8.0-3~18.04 1

[Zach Boyd (zboyd2)]

Removing the -dbg packages got me past that error, and it is currently doing all the upgrades. Thanks so much! Is there something special about -dbg packages that gets in the way of upgrading, or does it just happen that dist-upgrade was not smart enough to figure it out?

[Brian Murray (brian-murray)]

Reorganizing the replacments in _install_python_is_python2 (in DistUpgradeQuirks.py) so that python-minimal is first allowed the upgrade to be calculated with the apt-clone file in this bug and the one in bug 1915389.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Brian Murray (brian-murray)]

This should be resolved by the following change to ubuntu-release-upgrader.

ubuntu-release-upgrader (1:20.04.31) focal; urgency=medium

  * DistUpgrade/DistUpgradeQuirks.py: use apt's problem resolver to better
    calculate upgrades where python is replaced by python-is-python2 thereby
    resolving a host of upgrade failures. Thanks to Julian for the initial
    patch. (LP: #1898152)

 -- Brian Murray <email address hidden> Wed, 31 Mar 2021 10:13:14 -0700