Upgrading from lunar to mantic fails

[Robert Bamler]

When trying to upgrade from lunar to mantic, I originally got the following error message:

> An unresolvable problem occurred while calculating the upgrade.
>
> This was likely caused by:
> * Unofficial software packages not provided by Ubuntu
> Please use the tool 'ppa-purge' from the ppa-purge
> package to remove software from a Launchpad PPA and
> try the upgrade again.
>
> If none of this applies, then please report this bug using the command
> 'ubuntu-bug ubuntu-release-upgrader-core' in a terminal. If you want to
> investigate this yourself the log files in '/var/log/dist-upgrade' will
> contain details about the upgrade. Specifically, look at 'main.log' and
> 'apt.log'.

I tried the following steps to fix the issue:

- made sure my apt sources didn't contain PPA packages or external sources:

    egrep -v '^#|^ *$' /etc/apt/sources.list /etc/apt/sources.list.d/*
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar main restricted
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-updates main restricted
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar universe
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-updates universe
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar multiverse
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-updates multiverse
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-backports main restricted universe multiverse
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-security main restricted
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-security universe
    /etc/apt/sources.list:deb http://mirror.eu-fr.kamatera.com/ubuntu/ lunar-security multiverse

- made sure I didn't have any foreign repositories installed:

    grep Foreign /var/log/dist-upgrade/main.log
    2023-11-25 19:10:41,959 DEBUG Foreign:

- I remembered that `sudo apt upgrade` had shown some "held back"
  packages, so I installed all of them with
  `sudo apt-get --with-new-pkgs upgrade <list of held-back packages>`.
  Afterwards, `sudo apt upgrade` no longer showed any held-back packages.

When I now try to update to mantic, I get a slightly shorter error message:

> Could not calculate the upgrade
>
> An unresolvable problem occurred while calculating the upgrade.
>
> If none of this applies, then please report this bug using the command
> 'ubuntu-bug ubuntu-release-upgrader-core' in a terminal. If you want to
> investigate this yourself the log files in '/var/log/dist-upgrade' will
> contain details about the upgrade. Specifically, look at 'main.log' and
> 'apt.log'.

(including the space at the beginning of the last paragraph (before "If none of this ..."), which indicates to me that may there be should have been a list of possible reasons here but it just happens to be empty in my case)

ProblemType: Bug
DistroRelease: Ubuntu 23.04
Package: ubuntu-release-upgrader-core 1:23.04.7
ProcVersionSignature: Ubuntu 6.2.0-37.38-generic 6.2.16
Uname: Linux 6.2.0-37-generic x86_64
ApportVersion: 2.26.1-0ubuntu2.1
Architecture: amd64
CasperMD5CheckResult: pass
CrashDB: ubuntu
CrashReports:
 640:0:123:2808025:2023-11-25 13:15:52.076979189 +0100:2023-11-25 13:15:52.124979290 +0100:/var/crash/_usr_lib_bluetooth_bluetoothd.0.crash
 644:0:123:0:2023-11-25 13:15:52.076979189 +0100:2023-11-25 13:15:52.076979189 +0100:/var/crash/_usr_lib_bluetooth_bluetoothd.0.upload
 600:116:123:37:2023-11-25 13:15:53.364981874 +0100:2023-11-25 13:15:53.364981874 +0100:/var/crash/_usr_lib_bluetooth_bluetoothd.0.uploaded
CurrentDesktop: ubuntu:GNOME
Date: Sat Nov 25 18:56:36 2023
InstallationDate: Installed on 2022-03-10 (625 days ago)
InstallationMedia: Ubuntu 21.10 "Impish Indri" - Release amd64 (20211012)
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
UpgradeStatus: Upgraded to lunar on 2023-11-25 (0 days ago)
VarLogDistupgradeTermlog:

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robert Bamler (robamler)]

I managed to upgrade to mantic by removing all packages labeled as "broken" in /var/log/dist-upgrade/apt.log and reinstalling them (+ all packages that were removed because they depended on removed packages) after the upgrade. A few packages don't seem to exist in mantic anymore, so I skipped them (mostly around video encoding).

Unfortunately, this lead to 40 packages to be temporarily removed, including `gnome-shell`, so I didn't have a graphical user interface after the release upgrade. From an end-user's perspective, this is a really unfortunate situation, in particular because it's hard to understand why manual intervention is necessary here at all. Once I believed I had roughly figured out what the text in the log files meant, everything else was purely mechanical: extract the names of the packages that were identified as "broken", remove them, record the transitively removed packages, upgrade, and then reinstall the removed packages that were still available in the new release. Maybe ubuntu-release-upgrader could at least have proposed to do this for me.

[Kellen Renshaw (krenshaw)]

I ran into a similar, if not identical issue during a Jammy -> Lunar -> Mantic upgrade where Jammy to Lunar worked fine but the upgrade to Mantic failed with the error in this bug.

I was able to upgrade by removing the ubuntu-desktop, ubuntu-desktop-minimal, and ubuntu-minimal packages and reinstalling them after the upgrade to Mantic.

Attaching the contents of /var/log/dist-upgrade for that machine in case it helps.