KernelRemoval section of DistUpgrade.cfg shouldn't require manual updating

Bug #1828100 reported by Brian Murray on 2019-05-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
Status tracked in Eoan
Eoan
High
Brian Murray

Bug Description

Currently the method to determine kernels to auto remove uses a hard-coded version number and package base names in DistUpgrade.cfg. This need to be updated for every release of Ubuntu and has been missed in the past - bug 1828072. It'd be better if the autoremovalable kernels did not need to be updated all the time and instead utilized the logic apt uses (/etc/apt/apt.conf.d/01autoremove) to determine what kernels are removable.

Related branches

tags: added: rls-ee-incoming
summary: - KernelRemoval section of DistUpgrade.cfg requires manual updating
+ KernelRemoval section of DistUpgrade.cfg shouldn't require manual
+ updating
Brian Murray (brian-murray) wrote :

Additionally, the Version part of KernelRemoval is expected to be a string and not a list so you can't have multiple kernel versions thereby making it so that we can only remove the release kernel or the hwe one which is also lame.

tags: added: id-5cd1c9cc9dc3ea20e97b7fbd
Steve Langasek (vorlon) on 2019-05-09
Changed in ubuntu-release-upgrader (Ubuntu):
importance: Undecided → High
tags: removed: rls-ee-incoming
Balint Reczey (rbalint) wrote :

I'm wondering if we still need this functionality in u-r-u.

It offers removing removal of all autoremovable packages at the end of the upgrade for a long time and kernels are autoremoved periodically by unattended-upgrades and update-manager in Xenial and up.

IMO those are enough and since u-r-u performs autoremoval after the upgrade it is not affected by apt's LP: #1615381.

Changed in ubuntu-release-upgrader (Ubuntu Eoan):
status: New → Confirmed
Balint Reczey (rbalint) wrote :

So I propose just dropping the functionality from u-r-u which would solve the maintenance problem since the kernels are removed in other ways.

tags: added: id-5cd44c37cb4cfa83161e8d98
Brian Murray (brian-murray) wrote :

Testing an upgrade from Ubuntu 18.10 to Ubuntu 19.04 I noticed that the identifyObsoleteKernels function returns a set of linux packages which was already identified as obsolete so dropping the functionality as Balint suggested seems easiest.

Relevant part of /var/log/dist-upgrade/main.log:

2019-06-19 13:31:02,029 DEBUG Obsolete: gir1.2-mutter-3 libbind9-160 libdns-export1102 libdns1102 libedataserver-1.2-23 libexempi3 libgdbm5 libhunspell-1.6-0 libicu60 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 liblouis16 liblwres160 libmutter-3-0 libntfs-3g88 liborcus-0.13-0 libperl5.26 libpoppler79 libprotobuf10 libpython3.6 libpython3.6-minimal libpython3.6-stdlib libraw16 libreadline7 libssl1.0.0 linux-headers-4.18.0-10 linux-headers-4.18.0-10-generic linux-headers-4.18.0-18 linux-headers-4.18.0-18-generic linux-image-4.18.0-10-generic linux-image-4.18.0-18-generic linux-modules-4.18.0-10-generic linux-modules-4.18.0-18-generic linux-modules-extra-4.18.0-10-generic linux-modules-extra-4.18.0-18-generic perl-modules-5.26 python-talloc python3.6 python3.6-minimal
2019-06-19 13:31:02,029 DEBUG Foreign:
2019-06-19 13:31:02,029 DEBUG identifyObsoleteKernels()
2019-06-19 13:31:04,958 DEBUG removing obsolete kernel 'linux-headers-4.18.0-10-generic'
2019-06-19 13:31:04,958 DEBUG skipping running kernel linux-headers-4.18.0-18-generic
2019-06-19 13:31:04,961 DEBUG removing obsolete kernel 'linux-image-4.18.0-10-generic'
2019-06-19 13:31:04,961 DEBUG skipping running kernel linux-image-4.18.0-18-generic
2019-06-19 13:31:04,967 DEBUG removing obsolete kernel 'linux-modules-4.18.0-10-generic'
2019-06-19 13:31:04,967 DEBUG skipping running kernel linux-modules-4.18.0-18-generic
2019-06-19 13:31:04,969 DEBUG removing obsolete kernel 'linux-modules-extra-4.18.0-10-generic'
2019-06-19 13:31:04,969 DEBUG skipping running kernel linux-modules-extra-4.18.0-18-generic
2019-06-19 13:31:06,446 DEBUG identifyObsoleteKernels found '{'linux-modules-extra-4.18.0-10-generic', 'linux-modules-4.18.0-10-generic', 'linux-image-4.18.0-10-generic', 'linux-headers-4.18.0-10-generic'}'
2019-06-19 13:31:06,447 DEBUG forced_obsoletes: ['desktop-effects', 'gnome-app-install', 'policykit-gnome', 'gnome-mount', 'linux-modules-extra-4.18.0-10-generic', 'linux-modules-4.18.0-10-generic', 'linux-image-4.18.0-10-generic', 'linux-headers-4.18.0-10-generic']

Brian Murray (brian-murray) wrote :
Download full text (4.5 KiB)

I also tested an upgrade from Ubuntu 19.04 to eoan and here we can see identifyObsoleteKernels actually attempting to do something wrong - suggesting to remove the kernel just installed. While this is because Ubuntu 19.04 and eoan currently have the same kernel version it still seems like an argument for dropping the function.

Relevant part of /var/log/dist-upgrade/main.log:

2019-06-19 14:51:33,447 DEBUG Obsolete: libisl19 libpoppler85 linux-headers-5.0.0-13 linux-headers-5.0.0-13-generic linux-image-5.0.0-13-generic linux-modules-5.0.0-13-generic linux-modules-extra-5.0.0-13-generic
2019-06-19 14:51:33,448 DEBUG Foreign:
2019-06-19 14:51:33,448 DEBUG identifyObsoleteKernels()
2019-06-19 14:51:36,381 DEBUG skipping running kernel linux-headers-5.0.0-13-generic
2019-06-19 14:51:36,381 DEBUG removing obsolete kernel 'linux-headers-5.0.0-17-generic'
2019-06-19 14:51:36,383 DEBUG skipping running kernel linux-image-5.0.0-13-generic
2019-06-19 14:51:36,383 DEBUG removing obsolete kernel 'linux-image-5.0.0-17-generic'
2019-06-19 14:51:36,387 DEBUG skipping running kernel linux-modules-5.0.0-13-generic
2019-06-19 14:51:36,387 DEBUG removing obsolete kernel 'linux-modules-5.0.0-17-generic'
2019-06-19 14:51:36,387 DEBUG skipping running kernel linux-modules-extra-5.0.0-13-generic
2019-06-19 14:51:36,388 DEBUG removing obsolete kernel 'linux-modules-extra-5.0.0-17-generic'
2019-06-19 14:51:37,872 DEBUG identifyObsoleteKernels found '{'linux-modules-extra-5.0.0-17-generic', 'linux-image-5.0.0-17-generic', 'linux-headers-5.0.0-17-generic', 'linux-modules-5.0.0-17-generic'}'
2019-06-19 14:51:37,872 DEBUG forced_obsoletes: ['desktop-effects', 'gnome-app-install', 'policykit-gnome', 'gnome-mount', 'linux-modules-extra-5.0.0-17-generic', 'linux-image-5.0.0-17-generic', 'linux-headers-5.0.0-17-generic', 'linux-modules-5.0.0-17-generic']
2019-06-19 14:51:38,179 DEBUG Unused dependencies: pppoeconf libisl19 pppconfig libpoppler85 ifupdown
2019-06-19 14:51:38,179 DEBUG remove_candidates: '{'linux-modules-extra-5.0.0-13-generic', 'pppoeconf', 'gnome-app-install', 'libisl19', 'desktop-effects', 'linux-image-5.0.0-13-generic', 'pppconfig', 'linux-headers-5.0.0-13-generic', 'linux-modules-5.0.0-17-generic', 'linux-headers-5.0.0-17-generic', 'policykit-gnome', 'ifupdown', 'linux-headers-5.0.0-13', 'gnome-mount', 'linux-image-5.0.0-17-generic', 'linux-modules-5.0.0-13-generic', 'libpoppler85', 'linux-modules-extra-5.0.0-17-generic'}'
2019-06-19 14:51:38,179 DEBUG Start checking for obsolete pkgs
2019-06-19 14:51:38,180 DEBUG skipping running kernel pkg 'linux-modules-extra-5.0.0-13-generic'
2019-06-19 14:51:38,180 DEBUG 'linux-modules-extra-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 14:51:38,399 DEBUG skipping running kernel pkg 'linux-image-5.0.0-13-generic'
2019-06-19 14:51:38,399 DEBUG 'linux-image-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 14:51:38,508 DEBUG skipping running kernel pkg 'linux-headers-5.0.0-13-generic'
2019-06-19 14:51:38,508 DEBUG 'linux-headers-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 14:51:38,722 DEBUG package 'linux-module...

Read more...

Brian Murray (brian-murray) wrote :

I went ahead and disabled the identifyObsoleteKernels function and here is the relevant part of /var/log/dist-upgrade/main.log:

2019-06-19 15:37:29,707 DEBUG Obsolete: libisl19 libpoppler85 linux-headers-5.0.0-13 linux-headers-5.0.0-13-generic linux-image-5.0.0-13-generic linux-modules-5.0.0-13-generic linux-modules-extra-5.0.0-13-generic
2019-06-19 15:37:29,707 DEBUG Foreign:
2019-06-19 15:37:29,707 DEBUG forced_obsoletes: ['desktop-effects', 'gnome-app-install', 'policykit-gnome', 'gnome-mount']
2019-06-19 15:37:30,001 DEBUG Unused dependencies: libisl19 pppconfig ifupdown libpoppler85 pppoeconf
2019-06-19 15:37:30,001 DEBUG remove_candidates: '{'libisl19', 'linux-image-5.0.0-13-generic', 'gnome-mount', 'policykit-gnome', 'desktop-effects', 'linux-modules-5.0.0-13-generic', 'pppconfig', 'linux-modules-extra-5.0.0-13-generic', 'gnome-app-install', 'libpoppler85', 'ifupdown', 'linux-headers-5.0.0-13', 'linux-headers-5.0.0-13-generic', 'pppoeconf'}'
2019-06-19 15:37:30,001 DEBUG Start checking for obsolete pkgs
2019-06-19 15:37:30,100 DEBUG skipping running kernel pkg 'linux-image-5.0.0-13-generic'
2019-06-19 15:37:30,100 DEBUG 'linux-image-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 15:37:30,101 DEBUG skipping running kernel pkg 'linux-modules-5.0.0-13-generic'
2019-06-19 15:37:30,101 DEBUG 'linux-modules-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 15:37:30,198 DEBUG skipping running kernel pkg 'linux-modules-extra-5.0.0-13-generic'
2019-06-19 15:37:30,198 DEBUG 'linux-modules-extra-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 15:37:30,720 DEBUG skipping running kernel pkg 'linux-headers-5.0.0-13-generic'
2019-06-19 15:37:30,721 DEBUG 'linux-headers-5.0.0-13-generic' scheduled for remove but not safe to remove, skipping
2019-06-19 15:37:30,822 DEBUG Finish checking for obsolete pkgs
2019-06-19 15:37:30,861 DEBUG The following packages are marked for removal: linux-headers-5.0.0-13 ifupdown linux-headers-5.0.0-13-generic pppoeconf libisl19 pppconfig libpoppler85

So just removing the function seems like the best way forward.

Brian Murray (brian-murray) wrote :

It was suggested that testing upgrades from Ubuntu 18.04 should also be done so I tested an upgrade from Ubuntu 18.04 to Ubuntu 19.04 and eoan (which will become Ubuntu 19.10) and both of these behaved in the same way with the removal of identifyObsoleteKernels so lets do that.

Changed in ubuntu-release-upgrader (Ubuntu Eoan):
assignee: nobody → Brian Murray (brian-murray)
status: Confirmed → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:19.10.9

---------------
ubuntu-release-upgrader (1:19.10.9) eoan; urgency=medium

  * DistUpgrade/DistUpgradeCache.py, DistUpgrade/DistUpgradeController.py:
    drop code which identified obsolete kernels and marked them for removal as
    it is redundant with functionality that exists in apt. (LP: #1828100)
  * DistUpgrade/DistUpgradeQuirks.py: Create a fallback mechanism for
    determining the version of a release (necessary for Ubuntu 18.04) when
    replacing debs with snaps. (LP: #1833753)

 -- Brian Murray <email address hidden> Fri, 21 Jun 2019 14:30:19 -0700

Changed in ubuntu-release-upgrader (Ubuntu Eoan):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers