upgrades fail when package in ForcedObsoletes does not exist

[Nick Rosbrook]

[Impact]

With ubuntu-release-upgrader in noble-proposed, I get the following error when upgrading Ubuntu Desktop:

2024-08-19 14:48:13,087 ERROR not handled exception:
Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/apt/cache.py", line 281, in __getitem__
    # It might be excluded due to not having a version or something

KeyError: 'gnome-app-install'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/nr/24.04.22/./noble", line 8, in <module>
    sys.exit(main())

  File "/home/nr/24.04.22/DistUpgrade/DistUpgradeMain.py", line 241, in main
    if app.run():

  File "/home/nr/24.04.22/DistUpgrade/DistUpgradeController.py", line 2639, in run
    return self.fullUpgrade()

  File "/home/nr/24.04.22/DistUpgrade/DistUpgradeController.py", line 2615, in fullUpgrade
    self.doPostUpgrade()

  File "/home/nr/24.04.22/DistUpgrade/DistUpgradeController.py", line 2033, in doPostUpgrade
    if not self.cache[pkgname].marked_delete:

  File "/usr/lib/python3/dist-packages/apt/cache.py", line 283, in __getitem__
    raise KeyError("The cache has no package named %r" % key)

KeyError: "The cache has no package named 'gnome-app-install'"

This is because the new forced obsoletes code does not handle key errors with the cache.

[Test Plan]

The simplest way to test this is doing an upgrade of Ubuntu Desktop, because the DistUpgrade.cfg has a ForceObsoletes= section for `ubuntu-desktop` that contains at least one package that no longer exists in the archive.

$ do-release-upgrade -d

The upgrade should succeed past the "removing obsoletes" section.

[Where problems could occur]

The patch here is to check for the package name as a key in the cache before attempting to access it. Any more problems would be contained in the "searching/removing obsoletes" logic.

[Łukasz Zemczak (sil2100)]

Hello Nick, or anyone else affected,

Accepted ubuntu-release-upgrader into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:24.04.22 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Nick Rosbrook (enr0n)]

I verified the fix using ubuntu-release-upgrader 1:24.04.22 from noble-proposed:

nr@clean-jammy-amd64:~$ do-release-upgrade --proposed

[...SNIP...]

System upgrade is complete.

Restart required

To finish the upgrade, a restart is required.
If you select 'y' the system will be restarted.

Continue [yN] N
nr@clean-jammy-amd64:~$ grep forced_obsoletes /var/log/dist-upgrade/main.log
2024-08-22 13:00:06,173 DEBUG forced_obsoletes: ['desktop-effects', 'gnome-app-install', 'policykit-gnome', 'gnome-mount']

The non-existent packages in force_obsoletes did not cause a crash.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:24.10.8

---------------
ubuntu-release-upgrader (1:24.10.8) oracular; urgency=medium

  * DistUpgrade: handle cache key error when removing obsolete packages
    (LP: #2077358)
  * DistUpgradeQuirks: prevent upgrades if zfs module is loaded
  * deb2snap: switch snap-store and firmware-updater tracks (LP: #2077344)
  * Run pre-build.sh: updating mirrors, demotions, and translations.

 -- Nick Rosbrook <email address hidden> Fri, 23 Aug 2024 09:22:42 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:24.04.22

---------------
ubuntu-release-upgrader (1:24.04.22) noble; urgency=medium

  [ Tobias Heider ]
  * Fix typo in force_obsoletes. (LP: #2076913)

  [ Nick Rosbrook ]
  * DistUpgrade: handle cache key error when removing obsolete packages
    (LP: #2077358)
  * deb2snap: switch snap-store and firmware-updater tracks
    (LP: #2077344)
  * Run pre-build.sh: updating mirrors, demotions, and translations.

ubuntu-release-upgrader (1:24.04.21) noble; urgency=medium

  * DistUpgradeQuirks: disable stub resolver on upgrades from jammy
    (LP: #2043820)
  * DistUpgrade: show a message about denied package removals
    (LP: #2073278)
  * data: add cryptsetup-initramfs to removal_denylist.cfg
    (LP: #2067622)
  * DistUpgradeQuirks: abort if system is not /usr-merged
    (LP: #2058648)
  * DistUpgradeQuirks: disable cloud-init on upgrade
    (LP: #2075968)
  * DistUpgradeQuirks: abort if rabbitmq-server installed
    (LP: #2074309)
  * Run pre-build.sh: updating mirrors, demotions, and translations.

ubuntu-release-upgrader (1:24.04.20) noble; urgency=medium

  [ Mate Kukri ]
  * DistUpgradeQuirks: make sure GRUB "cloud_style_installation" is set in cloud images (LP: #2054103)

  [ Julian Andres Klode ]
  * Move the actiongroup out of the obsoletes loop
  * Reimplement obsolete removal with resolve_by_keep (LP: #1874272)

  [ Nick Rosbrook ]
  * Run pre-build.sh: updating mirrors, demotions, and translations.

 -- Nick Rosbrook <email address hidden> Wed, 21 Aug 2024 12:55:15 -0400

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.