Kernels not autoremoving, causing out of space error on LVM or Encrypted installation or on any installation, when /boot partition gets full

Bug #1357093 reported by Elfy on 2014-08-14
694
This bug affects 212 people
Affects Status Importance Assigned to Milestone
unattended-upgrades
New
Undecided
Unassigned
unattended-upgrades (Ubuntu)
Wishlist
Unassigned

Bug Description

Currently if one chooses to use LVM or encrypted install, a /boot partition is created of 236Mb

Once kernel updates start being released this partition soon fills until people are left unable to upgrade.

While you and I might know that we need to watch partition space, many of the people we have installing think that a windows disk is a disk and not a partition, education is probably the key - but in the meantime support venues keep needing to deal with the fact the partition is too small and/or old kernels are not purged as new ones install.

For workaround and sytem repair, see
https://help.ubuntu.com/community/RemoveOldKernels

Elfy (elfy) wrote :
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubiquity (Ubuntu):
status: New → Confirmed
Elfy (elfy) wrote :

Installed with LVM in a vm, /boot size 236M

/dev/sda1 236M 38M 186M 17% /boot

http://pastebin.ubuntu.com/8062443/

Installing with encryption and LVM results in the same /boot size unsurprisingly

description: updated
description: updated
summary: - Encrypted install creates too small /boot partition
+ LVM or Encrypted install creates too small /boot partition

Just dealt with the problem of an affected user here : http://ubuntuforums.org/showthread.php?t=2240240

But I guess adding myself to the "Affects Me" list would be cheating? Since it is not me who is affected.

TJ (tj) wrote :

The issue here isn't so much the absolute space initially allocated to the /boot/ LV, it is that the installer allocates 100% of the Volume Groups space.

The great advantage of LVM is that space can by dynamically allocated and removed from Logical Volumes as needed.

If the installer were to leave 5% of the Volume Group unallocated it would be possible for the system to detect this out-of-space condition and extend the /boot/ LV:

lvextend -l 50%FREE VG/boot
resize2fs /dev/mapper/VG-boot # assuming ext file-system

and at the same time display a prominent warning and options for automatically removing the superseded kernel versions.

Especially a problem when setting up systems for clients, who you only visit once or twice a year. The kernel updates fill the boot partition fast and so users are warned at the start that no space is left.

Donald Tripp (dtlongrifles) wrote :

I have this problem with Xubuntu 14.04

Changed in ubiquity (Ubuntu):
assignee: nobody → Donald Tripp (dtlongrifles)
Coffeecat (coffeecat) on 2014-12-13
Changed in ubiquity (Ubuntu):
assignee: Donald Tripp (dtlongrifles) → nobody
Sergey Romanov (sml-uni) wrote :

Link to yet another forum thread: http://ubuntuforums.org/showthread.php?t=2256479
The problem is twofold.

1) Users not paying attention to apt-get saying:

The following packages were automatically installed and are no longer required:
...
Use 'apt-get autoremove' to remove them.

like in the thread linked above.

2) Systems where linux-image-<version>-{generic,server,goldfish,lowlatency} don't get suggested for autoremoval after upgrade to newer versions of linux-{generic,server,goldfish,lowlatency}/linux-image-{generic,server,goldfish,lowlatency}. Mostly because some dkms-based kernel-modules are installed like NVIDIA/AMD propietary video drivers or VirtualBox. dkms recommends virtual package linux-image provided by each and every linux-image-<version>-{generic,server,goldfish,lowlatency}. That prevents their autoremoval as long as dkms is installed and apt is configured not to autoremove recommended packages (Apt::AutoRemove::RecommendsImportant).

Sergey Romanov (sml-uni) wrote :

Correction.
APT::AutoRemove::RecommendsImportant and APT::AutoRemove::SuggestsImportant are only used by aptitude. apt-get just recycles the values of APT::Install-Recommends and APT::Install-Suggests when deciding which packages should be autoremoved. apt-get's behaviour is also affected by an undocumented and unused option APT::Install-Recommends-Sections that allows autoinstalling/autoremoval of Recommends to be restricted to certain archive sections.

Nevertheless, since APT::Install-Recommends is usually set to true on Ubuntu, the point 2) from my previous comment still applies.

Cornelis Spronk (spronkc) wrote :

boot partition fills up with updates of kernel.

autoremove does not remove previous kernels

jsland (greenplanet) wrote :

Same problem here for some months now on 14.04 Studio. Autoremove and bleachbit don't deal with it. Manual cleanup around the November timeframe last year got me to the March cycle of updates before /boot was overloaded again. Problem appears to me to be centered on the fact the old ver. are not being removed completely. Using synaptic to get rid of older ver. only seems to clear a fraction of the reported image size.

Sam (litle-sam) wrote :

Added my +1 as it affects me to.

remerson (ubuntu-remerson) wrote :

This bug is tedious. /boot holds only 3 kernels for me and it seems there is a new kernel every couple of weeks. So, each time, I need to go and delete the previous 1-2 manually. Why do I have to babysit my machine? Lucky I do not want to do unattended upgrades.

Resizing /boot with LVM requires much hoop-jumping (good luck for a beginner) and only slows down how often this happens.

Please can old kernels be auto-removed properly (And/or /boot sized a bit more sensibly by default?).

There is a "latest" kernel metapackage. Can we have a "previous1" and/maybe "previous2" as well - then everything else can just go, right?

Wylbur (wilbur-wilbur) wrote :

Increasing the boot partition does not solve the underlying problem of old kernels. Is the size large enough now to accommodate enough kernels? If it is, we should tackle the real problem of removing old kernels.

Snoopy (snoopy-u) wrote :

Same problem as indicated above on 14.04 LTS. Why are "old" kernels left on a drive that is too small to begin with? Thank you in advance for fixing it. If fix would allow to expand partition (at least double it) and remove "old" kernels, this would be great!

Cazacu Bogdan (cazacub) on 2015-06-22
information type: Public → Public Security
information type: Public Security → Public
James Donner (jamesadonner) wrote :

Trying this on a Dell Inspiron 15 7547 running 100% Ubuntu (No dual booting). We're unable to upgrade from Ubuntu 14.10 to 15.04 due to this error. We've tried

>sudo apt-get clean
>sudo apt-get autoremove

and nothing has worked. We don't want to get left behind on software updates, so fixing the problem is a high priority.

Hedley Finger (hedley-finger) wrote :

I have this problem but removing old kernels was only partly successful, probably because I used rm and not apt-get method. In fact, if you have a laptop, why use LVM at all? You are scarcely going to wander around with an external hard disk attached to it for the extension to a logical volume.

The installer needs to be a lot smarter about people's level of knowledge re partitions and kernels, and give options for adjusting some of the parameters, e.g. LVM v. normal partitions, size of /boot, guided removal of old kernels, perhaps keeping three consecutive updates (including the current working kernel).

I just went through a torrid attempt to use Gparted (doesn't recognise LVM lv's) and system-config-lvm (which shows the swap and boot partitions at one end of the sda5 red cylinder, and the logical view, which shows them at the OTHER end of the blue cylinder).

I am totally lost in techhead space so am resorting to a new install to "fix" this problem. I chose Ubuntu over Fedora, SuSE and all the rest because it seemed the most novice-friendly, but this ordeal has proved me wrong. 8^)

Remun.J (remun-j66) wrote :

Someone give me the advise to use the LVM option on a fresh Xubuntu 15.04 install so i did. Use one internal drive, a 256GB SSD and a 2TB network drive (small NAS) for external storage. Hope there will be a fix to autoremove those old kernels from /boot partitions before upgrading a new one.

same here, Ubuntu 15.04 64 bit, encrypted [as opposed to the LVM].
Installed about 2015/09/15, and the Update Manager refuses installing already the very first update to the Kernel package(s), i.e., from 3.19.0.28.27
to 3.19.0.30.29.

·

·

·

[I know it's off-topic here, but is there any way round except reinstalling of the whole system? Which btw I know now I have to do manually :) ]

georgebaily (z-george) wrote :

Firstly, me too, this is super annoying. As a rule of thumb any 'feature' that *requires* the user to drop to CLI to put in regular manual commands *to keep the system secure and updated* ... in a desktop / consumer OS... is a bug.

It seems there are separate issues here that could be addressed separately:

1. the auto setup does not intelligently ask/warn about its default of setting up a very small /boot

2. there exists no user-friendly auto cleanup for the things clogging up /boot

3. the /boot does not have any elegant way of automatically (or via a setting) "overflowing" into the (usually abundant) storage space outside of /boot for its old kernel data etc

4. when the "full, cannot update any more" problem occurs, there is no useful feedback or suggestion or wizard from the OS for the user... just essentially a FU message.

5. apparently there is not an "easy" way to resize /boot to allow more crap to accumulate and thus postpone the problem... if there *is* an easy way then see (4) it should be provided then and there to the user; if there is *not* an easy way then this proves the point that this is not a trivial bug.

Addressing at least one of these would help the problem, and all could be worked on.

Ian Weisser (ian-weisser) wrote :

apt-get remove/purge/autoremove is blocked by the (always-failing) kernel install (not enough space on device)

affects: ubiquity (Ubuntu) → apt (Ubuntu)
Ian Weisser (ian-weisser) wrote :

Ubiquity creates space for enough kernels. The system is not managing that space later. Not ubiquity's fault.

These two issues together create the problem on a small /boot partition:
apt: apt-get remove/purge/autoremove is blocked by the (always-failing) kernel install (not enough space on device)
unattended-upgrade: 'autoremove' is disabled by default.

Adding both packages to the bug report. Removing ubiquity.

Workaround - How users can recover from the problem:
- You can use dpkg to manually remove older kernels until apt has enough space to work, or
- You can use dpkg --set selections to un-select the kernel packages for install, then run apt-get autoremove

Workaround - How users can prevent recurrence after successful recovery:
- You can enable autoremoval in /etc/apt/apt.conf.d/50unattended-upgrades , or
- You can mark your wall calendar and manually run apt-get autoremove every few months

summary: - LVM or Encrypted install creates too small /boot partition
+ Kernels not autoremoving, causing out of space error on LVM or Encrypted
+ install with small /boot partition

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in unattended-upgrades (Ubuntu):
status: New → Confirmed

Change summary, because /boot is always to small on "LVM or Encrypted install". The former summary read like it was a user error, while it isn't.

summary: Kernels not autoremoving, causing out of space error on LVM or Encrypted
- install with small /boot partition
+ install
Ian Weisser (ian-weisser) wrote :

Developer of unattended upgrades has submitted a patch:
https://github.com/mvo5/unattended-upgrades/pull/19

Changed in apt (Ubuntu):
status: Confirmed → In Progress
Changed in unattended-upgrades (Ubuntu):
status: Confirmed → In Progress
no longer affects: unattended-upgrades (Ubuntu)
affects: apt (Ubuntu) → unattended-upgrades (Ubuntu)
Achim Behrens (k1l) wrote :

This really needs to be adressed ASAP.

The Support Communites are flooded by angry users with blocked Packagesystems due to a full /boot blocking new kernels to be installed.

In this time, when we have 5TB HDDs and even 500GB SSDs, i really dont know why we still need that small /boot partitions that it runs out of space that fast.
And having no automatic removing old kernels is another issue.

This really spoils the user experience, and non technical users really dont know what to do and dont feel comfortable to delete files in /boot manually. So best is do increase the /boot partition made on install and have a post-install script running that removes old kernels.

Please do not spam bug report with "me too!" or "this bug is awful!"
comments.

The bug report is a living document used by the triagers and developers
to share data, many of them volunteers, and noise makes their job
harder.

A fix has has already been made in the upstream git, and is awaiting
merge. See https://github.com/mvo5/unattended-upgrades/pull/19

There are a number of bugs marked duplicates of this bug that really aren't. The size of /boot is orthogonal to the issue of purging unused kernels. 256 MB gives the ability to install approximately 4 kernels. For some of us that's not enough, and we need to specify a larger /boot. Shortly, I intend to unmark 1465050 as a duplicate, and mark 1511911 as a duplicate of that.

Jarno Suni (jarnos) on 2016-01-11
summary: Kernels not autoremoving, causing out of space error on LVM or Encrypted
- install
+ installation or on any installation, when /boot partition gets full
Jarno Suni (jarnos) wrote :

Ian Weisser, how is Unattended-Upgrade::Remove-New-Unused-Dependencies better than Unattended-Upgrade::Remove-Unused-Dependencies that is already available? Why not just use a specific script to remove old kernels instead of using apt-get autoremove?

daniel smith (opened-to) wrote :

this is affecting me too.

Filesystem Size Used Avail Use% Mounted on
udev 981M 4.0K 981M 1% /dev
tmpfs 200M 1.1M 199M 1% /run
/dev/mapper/ubuntu--vg-root 72G 7.4G 61G 11% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 997M 404K 996M 1% /run/shm
none 100M 52K 100M 1% /run/user
/dev/sda1 236M 211M 14M 95% /boot
/home/rebecca/.Private 72G 7.4G 61G 11% /home/rebecca

i thought that it was because my drive was full. i think its only maybe 40 gig?
thanks

Will the fix, that has been merged in the meantime make it into 14.04 LTS or do we have to wait for some future Ubuntu release?

Jarno Suni (jarnos) wrote :

Fixing the issue using unattended upgrades in 14.04 LTS requires fixing Bug #1492709, too.

Jarno Suni (jarnos) wrote :

And it still would require manually removing the kernels installed before the fix is installed.

Ian Weisser (ian-weisser) wrote :
Changed in unattended-upgrades (Ubuntu):
status: In Progress → Fix Committed
Ian Weisser (ian-weisser) wrote :

Fix included in upstream release: unattended-upgrades 0.89 30 JAN 2016
Uploaded to Ubuntu 16.04 on 02 FEB 2016.

   * Add `Unattended-Upgrade::Remove-New-Unused-Dependencies` that
     defaults to "yes". This ensures that older kernel
     get automatically cleaned up and /boot will not overflow.
     LP: #1267059

Marking this bug as a duplicate of LP: #1267059 so it gets closed automatically when appropriate.

Jarno Suni (jarnos) wrote :

LP: #1267059 is about automatic removing fails even if
Unattended-Upgrade::Remove-Unused-Dependencies "true";
It was fixed already in unattended-upgrades - 0.76ubuntu1.2. There is an error in the changelog of unattended-upgrades. This is not a duplicate of bug #1267059. This bug is about making the autoremoval of excessive the default behavior.

Jarno Suni (jarnos) wrote :

Can someone attach the current default /etc/apt/apt.conf.d/50unattended-upgrades of ?Ubuntu 16.04 (clean install)? Is there "Remove-New-Unused-Dependencies" advertised in comment #36? Is there line
Unattended-Upgrade::Remove-Unused-Dependencies "true";
there and is it uncommented? What is in the Unattended-Upgrade::Allowed-Origins block? Is unattended-upgrades enabled by default: What does Updates-tab show in Software & Updates (i.e. software-properties-gtk), particularly items "When there are * updates". What does "sudo dpkg-reconfigure unattended-upgrades" show?

I upgraded from 15.10 i.e. not clean install, and there is no "Remove-New-Unused-Dependencies" in /etc/apt/apt.conf.d/50unattended-upgrades.

Jarno Suni (jarnos) wrote :

50unattended-upgrades does not contain mention for Unattended-Upgrade::Remove-New-Unused-Dependencies, but the default value is true. Automatic installing of security updates is enabled by default in Xenial.

Jarno Suni (jarnos) wrote :

Fixed in version 0.90

Changed in unattended-upgrades (Ubuntu):
status: Fix Committed → Fix Released
Jarno Suni (jarnos) on 2016-06-07
description: updated
Changed in unattended-upgrades (Ubuntu):
importance: Undecided → High
tags: added: precise trusty xenial
tags: removed: xenial
Changed in unattended-upgrades (Ubuntu):
importance: High → Wishlist
48 comments hidden view all 127 comments
Mossroy (mossroy) wrote :

Ian, could you please give the URLs where to find the "similar-looking apt problems in the support forums" you're mentioning?
I think many people here (including me) would be interested in what are the correct settings you mention.

Ian Weisser (ian-weisser) wrote :

Happy to discuss and help...in the support forums.
This CLOSED bug report is not the appropriate location to help people troubleshoot.

Mitch Singler (msingler) wrote :

Seems to be a design flaw rather than a bug.

      From: Ian Weisser <email address hidden>
 To: <email address hidden>
 Sent: Tuesday, November 22, 2016 9:26 PM
 Subject: [Bug 1357093] Re: Kernels not autoremoving, causing out of space error on LVM or Encrypted installation or on any installation, when /boot partition gets full

Happy to discuss and help...in the support forums.
This CLOSED bug report is not the appropriate location to help people troubleshoot.

--
You received this bug notification because you are subscribed to a
duplicate bug report (1183692).
https://bugs.launchpad.net/bugs/1357093

Title:
  Kernels not autoremoving, causing out of space error on LVM or
  Encrypted installation or on any installation, when /boot partition
  gets full

Status in unattended-upgrades package in Ubuntu:
  Fix Released

Bug description:
  Currently if one chooses to use LVM or encrypted install, a /boot
  partition is created of 236Mb

  Once kernel updates start being released this partition soon fills
  until people are left unable to upgrade.

  While you and I might know that we need to watch partition space, many
  of the people we have installing think that a windows disk is a disk
  and not a partition, education is probably the key - but in the
  meantime support venues keep needing to deal with the fact the
  partition is too small and/or old kernels are not purged as new ones
  install.

  For workaround and sytem repair, see
  https://help.ubuntu.com/community/Lubuntu/Documentation/RemoveOldKernels

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1357093/+subscriptions

Mossroy (mossroy) wrote :

For those still having this issue on Ubuntu 16.04 : please check that you did not disable the automatic installation of updates.
It might be the reason why the old kernels were not removed for me (not 100% yet). See discussion in https://ubuntuforums.org/showthread.php?t=2344232
With a GUI, it's in System settings->Software and updates->Updates tab : "When there are security updates" should be set to "Download and install automatically" (its default value).
(please comment on the forum thread)

Henry (hvanmegen) wrote :

I can't even run `apt-get autoremove` because it starts filling up my /boot with old kernels until it runs out of space.. who the hell designed this madness?

Henry (hvanmegen) wrote :

Ok, I had to manually remove the files as they were added.. after that, it worked as intended.

Funny thing is, all solutions provide here say that you need to enable the removal of unused dependencies by setting this:

`Unattended-Upgrade::Remove-Unused-Dependencies "true"`

Problem is.. I already had this set in my `/etc/apt/apt.conf.d/50unattended-upgrades`..

This is broken, design or not.. it IS a bug.. how can we fix this permanently? I have servers running unattended upgrades that have overflowing /boot partitions which is annoying AF.

Jarno Suni (jarnos) wrote :

Henry, do you use Ubuntu Trusty, and install kernels manually? See comment #79.

Bryce Nesbitt (bryce2) wrote :

My elderly parents are on Ubuntu 16.04, with updates set to 'Download and install automatically'.
I must manually prune the Kernel images every so often, or the system's underwear gets all in a twist, and "apt-get autoremove' won't even work.

Let's be clear: THIS IS NOT FIXED as of 16.04 LTS.

spike speigel (frail-knight) wrote :

This might be aggravated by this DKMS bug which was found in v2.2.0.3, which incidentally is used in Ubuntu 16.04 LTS, and maybe other versions. It doesn't remove old initrd files in /boot *which lead to full /boot issues and subsequent update problems*

I manually removed a bunch of old initrd images the other day.

http://askubuntu.com/questions/616512/purging-old-kernels-fails-to-remove-old-initrd-files

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717584

James Duvall (jamesduvall) wrote :

running 16.10, I am also running in to the problem that autoremove fails to remove old initrd files. This was greatly exacerbated by bug #798414 (https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/798414). With an encrypted file system and a small boot partition, I only have space for 3 kernels. With the pace of new kernel builds, it doesn't take long to run out of space. Now that I understand the problem, it is not such a big deal, but it was very annoying to have unhelpful initramfs errors nearly every time I boot up.

Jackalux (heartwoodjack) wrote :

I don't have hardware that I imagine will run 16.xx so need this addressed on older LTS versions.

My output for...
dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d'
 ... was 120 lines long! I had to teach myself scripting to delete them all manually. I only discovered the error after a different Linux machine I created (with a separate /boot) ran into the bug.

Deleting this junk freed 10GB of total disk space.

This is too much to ask of novice users and will stop Linux being more widely adopted. I'm not a complete Noob but this was annoying to deal with... it will be impossible for others.

Andreas Lindhé (lindhe) wrote :

I hate to be that guy that complains without contributing in a constructive manner, but I just need to say that it's completely unreasonable that my mom and friends I recommend Ubuntu to gets this problem. They should never have to be concerned with what a kernel image is or why the boot partition is full.

Jarno Suni (jarnos) wrote :

Jackalux, oh I see you have found the popular one-liner script for kernel removing. It is too complicated for what it does and too simple for what it should possibly do.

I have written a script to ease kernel removal:
https://www.bountysource.com/issues/38300038-feature-request-the-command-should-work-like-this
You (and other complainers) could post a small bounty to fund the work. Alternatively I think we could form an association to fund the work and other stuff that novice Ubuntu users may need provided it could attract enough paying members.

Andreas Lindhé, you have a point. I don't know who decides it, but I suppose the fix to the bug could be backported or SRUed to even older LTS releases. But it seems that the fix does not work even in 16.04 (Bug #1675079); that applies only to the new Unattended-Upgrade::Remove-New-Unused-Dependencies setting in unattended-upgrades, though, so if setting Unattended-Upgrade::Remove-Unused-Dependencies "true" suits to you, there is a fix available. Alternatively, e.g. my script could be run from a cron job or (in 16.04 or later) from a systemv timer to handle automatic kernel removal.

Jarno Suni (jarnos) wrote :

Correction: I meant systemd, not systemv in the previous comment.

cmp (cmp001) wrote :

I had previously recommended (and consequently installed for them) Ubuntu 16.04 LVM to my parents however their"/boot" partition was constantly filling up to 100% and they kept complaining to me about it (as they are computer illiterate) so I had to eventually reinstall windows 10 for them and scrap Ubuntu all together. A pretty sad state of affairs. This is definitely a bug and it is making people turn away from Linux in favor of Windows. I wish someone would provide a simple fix for this.

Jarno Suni (jarnos) on 2017-07-30
description: updated
Andreas Lindhé (lindhe) wrote :

Is this bug confirmed resolved for any version of Ubuntu? What release do I need to get my friends migrated to, in order for this to not be a problem?

Jarno Suni (jarnos) wrote :

No. Automatic removing is supposed to work in 16.04 and later by default. But even in 16.04 it works only, if you let unattended-upgrades install all kernel updates. I have not tested later releases, but I guess it is the same thing.

I think removing should work in 16.04 and later, if you use the configuration recommended in https://help.ubuntu.com/community/RemoveOldKernels. (i.e. setting Unattended-Upgrade::Remove-Unused-Dependencies "true";) This configuration works by 14.04, too, if you let unattended-upgrades install all kernel updates.

Stuart Bishop (stub) wrote :

It also requires a complete reinstall if your boot partition is too small (ie. you accepted the defaults with an LVM or encrypted disk install pre 16.04 release). Machines upgraded from 14.04 will continue to fail and be directed here.

Dave Eckhardt (de0u) wrote :

I tried the advice given on https://help.ubuntu.com/community/RemoveOldKernels, but it mostly doesn't work. For example, "apt-get autoremove --purge" removed zero kernels. To install "purge-old-kernels" would require installing like 60 dependencies of "bikeshed", which seems unreasonable.

I think part of the problem is that the tools behave differently depending on who installed a kernel, but /boot gets full based on the number of kernels, not who installed them. My machine (a netbook) is basically never left on long enough to run an unattended upgrade, so I run upgrades manually. That means basically every kernel was manually installed, so something that removes only automatically installed kernels is 0% useful. In the real world the problem is disk space, not assigning blame for who installed a given kernel! Any process that installs new kernels without deleting old kernels will eventually fill up a finite-sized /boot. Any solution that doesn't address the sad fact of the finiteness of /boot will leave some people, probably the most inexpert, stuck. I personally can manually grub around and remove kernels, so this is only annoying, not show-stopping, but I think we have lots of evidence that it is show-stopping for some people, which suggests that there are many more people out there who don't know how to complain who are also injured.

Bottom line: /boot is finite, so anybody who installs a kernel without deleting a kernel is doing something that will sink somebody's ship. Given that lots of machines out there are clogged by the old maintenance processes, any solution that doesn't explicitly address how to un-clog a machine clogged by the previous solutions isn't really a solution.

Jarno Suni (jarnos) wrote :

stub,

I suppose the upgrading process did not change the settings concerning unattended-upgrade. You should do that manually after upgrading from 14.04 and initially remove extra kernels manually before upgrading.

de0u,

I tell about the restrictions concerning 'apt-get autoremove' in the wiki page. Likewise I tell about some alternatives not limited to 'purge-old-kernels'. I think it is harsh to say most things do not work, if you do not specify.
You could run Unattended-upgrade from command line to install new kernel updates when you want:

sudo apt-get update && sudo unattended-upgrade

As for un-clogging, did you read the 'Safely Removing Old Kernels' chapter? I see many people take time to complain, but I see none who is willing to spend even 1$ to get an easier solution for poor Ubuntu users.

David Potter (akxw-hzxb-j0p9) wrote :

I'm running Ubuntu 17.04 and whilst I have had the problem for so long I can't be sure, I think I have had this problem since 15.04. I keep my software completely up to date and all updates have been applied to date. I can honestly say that this issue is by no means fixed. The reference to unattended-upgrades is a red herring. I boils down to a faulty kenal cleanup routine and a poor default for the boot partition at install time.

I have over the many months of experiencing the problem, finally come up with the following work around set of commands that free up the space required to keep me going with a few more updates...

sudo apt-get autoremove;sudo apt-get autoclean;sudo apt-get clean

dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

sudo apt-get update && sudo apt-get upgrade

Although this sorts the problem out when I get the "not enough space" issue it doesn't mean that the underlying problem shouldn't be sorted out.

It takes time and effort that shouldn't be required.
It has taken many months to come up with a decent work-around.
I can't use this type of install for family and friends as they are not savvy enough to use the work-around and shouldn't need to be using sudo in the first place.

I see this issue is marked as wishlist. This is a travesty. It is a bug pure and simple.

alias remove-old-kernels='dpkg -l linux-{image,headers}-* | awk '\''/^ii/{print }'\'' | egrep '\''[0-9]+\.[0-9]+\.[0-9]+'\'' | grep -v 4.4.0-79 | xargs sudo apt-get -y purge'

Dave Eckhardt (de0u) wrote :

jarnos,

The reason why I responded first/primarily about "apt-get autoremove" and "purge-old-kernels" is that those are listed at the top of the page and are the things that a regular-ish user could feel confident doing.

I do not feel that the idea of running unattended-upgrade from the command line is likely to be helpful to many users. Among other things, that presumes that the user always gets in there before being prompted by the automatic update GUI program (which, by the way, doesn't say "I am going to install a kernel"; it says "I have 192 MB of security updates that I would like you to approve", unless you dive into details).

As for un-clogging, I did in fact read the "Safely Removing Old Kernels" text. What I wrote was "I personally can manually grub around and remove kernels, so this is only annoying, not show-stopping, but I think we have lots of evidence that it is show-stopping for some people". I continue to believe that.

I did not see a response from you to what I genuinely believe is the single core issue: anything (unattended-upgrades, GUI install widget, apt, flying spaghetti monster) that installs a kernel without deleting a kernel will eventually overflow /boot.

Then there is a second issue, which is that because a variety of things have installed too many kernels on the machines of people who cannot be expected to manually delete kernels, there should be some automatically-run solution (perhaps prompting the user for confirmation for safety purposes) that deletes long-stale kernels that no current automatic thing considers itself responsible for.

What I see is:
(1) new users keep showing up because their machines have broken,
(2) people keep proposing new solutions that are different than the existing solutions.

To me this says that there is still a real problem (for some people, not for me, but I don't think the aspiration of the Ubuntu project is for machines running Ubuntu to work only for expert users).

Thanks for taking the time to respond! I am not responding because I am personally troubled by this issue, but because I believe that the problem has not been solved for regular end users.

Mitch Singler (msingler) wrote :

Well said Dave! It's hard for me to understand why smart people don't understand this issue. This required maintenance should not even be required for expert users let alone the average user. In over 30 years in IT and multiple platforms I have never heard of this design flaw/systems implementation being done on purpose. Perhaps someone can explain the design purpose that creates this issue as I have not seen it any where in these forums and bug reports.

Andreas Lindhé (lindhe) wrote :

I'm completely with Dave on this one.

Regarding jarnos comment #110:

> I see many people take time to complain, but I see none who is willing to spend even 1$ to get an easier solution for poor Ubuntu users.

Is funding the problem here? Because if it is, please tell us. I'm sure we can at least gather some funds, since this is such a prevalent issue that clearly stirs a lot of people into motion.

Just one quick comment that issue for me never got fixed, and i switched to debian. and will finally hopefully find a way to stop getting notified about comments on this thread.

Jarno Suni (jarnos) wrote :

The reason why 'apt-get autoremove' is stated there is that it is a simple working way in 16.04 and later for most users, if your system is not broken already. purge-old-kernels is included in some Ubuntu package; consequently it is supported by Launchpad's bug reporting system. But as said in the document, it is currently deprecated and its bugs will likely not be fixed. It might still be useful for some users in some cases.

If one wants to use unattended-upgrades manually, he/she could disable it from being called automatically. One should avoid installing security updates manually by Software Updater in 14.04, if one expects to use apt/unattended-upgrades for removing kernels automatically.

I have developed some software that makes the un-clogging part easier; link to related bountysource.com page is included in the document. Just feel free to post a bounty and/or give some feedback about the interface in the respective Launchpad page. I guess the software could be launched automatically in case of kernel clog when it happens in interactive session (or even in non-interactive session, if the software is made to support completely non-interactive un-clogging).

If you install a kernel manually in command line by say
sudo apt-get install linux-image-4.4.0-87-generic
which kernel(s) that command should remove in turn in your opinion? Or could it be that the user who runs the command should take care of it? User possibly wants to keep some spare kernels, in case the one that is just installed is bad. Or a developer may want to keep some testing kernels etc.

David Potter (akxw-hzxb-j0p9) wrote :

Hi Jarno

I would suggest that most desktop users get notified by the updater that there are updates and run the updater accordingly to apply them. As such, I think the unclogging activity needs to run automatically as the users that are using the default boot partition size almost certainly have no interest in having to run the software themselves or even making a decision about how many old kernels to keep. In fact I suggest that there are a goodly number of users that wouldn't know what to do with an old kernel anyway.

My suggestion would be that there should be a default number of kernels to keep (2 or maybe 3 would be my vote) but have a configuration file that would allow the number to be altered for those users that wanted to change the default. In fact you could even use a 0 value to indicate that the user does not want the un-clogging to happen automatically.

It seems like all the commands and scripts have been sorted out over time and the work is really to apply them in an automated fashion. I do recognise that, there may be a more elegant solution that might be preferable though. In which case that is going to be more work.

If we are talking about putting this in place to run automatically and definitely removing the "out of space" issue with no effort on behalf of the user, then I would be happy to contribute towards the bounty. Can you confirm that this would be the intention?

I don't think that the default size of the boot partition should be ignored though. Surely it's not going to break anyone's system to up the size to 500Mb. Whilst I'm not a developer, I can't believe it would take much effort to increase the default size.

Thanks for your persistence and help moving this bug along Jarno.

Jarno Suni (jarnos) wrote :

Hi David

One option is to install 16.04 or later and configure unattended-upgrades as recommended in the document. I am afraid all the related bugs will not be fixed in 14.04.

Fixing Bug #1460396 and Bug #291342 would also help users of Software Updater even if unattended-upgrades is not used.

The changes you suggest have nothing to do with unattended-upgrade (against which this bug is reported).

IIRC the default separate /boot size is 500MB in some newer release of Ubuntu.

Jarno Suni (jarnos) wrote :

As for default settings for unattended-upgrade, it is interesting what Canonical Foundations Team is going to do with Bug #1624644.

1 comments hidden view all 127 comments
Jarno Suni (jarnos) wrote :

I wonder why fix to Bug #1439769 has not been released as SRU for 14.04 yet. The fix would make unattended-upgrade work better in removing extra kernels in 14.04.

Jarno Suni (jarnos) wrote :

... and 'sudo apt-get autoremove' would work, too.

Ben Norris (norro) wrote :

This bug has just hosed a system I installed for my parents. An unattended-upgrader that breaks your system if you don't attend to it IS broken.

Having known about this problem for many years, the least you could have done is added a warning to the installer that choosing lvm or encryption will make a system that breaks itself.

The idea that novice users can just run script workarounds is entirely unrealistic.
a) They don't know about the problem if they haven't been told (burying it deep in the manuals or text that whizzes by is not good enough)
b) from a security PoV they shouldn't be run anything they cut & paste in

The general attitude to a major showstopper that is causing a lot of pain for novice users shown in this bug is frankly appalling and I'm pretty much inclined to ditch Ubuntu because of it.

iGadget (igadget) wrote :

Ben, I feel your pain and couldn't agree more. I'm under the impression that this attitude is getting more common as Canonical focuses on server and IoT. Where are the days where the 100 Papercuts (https://wiki.ubuntu.com/One%20Hundred%20Papercuts) got a really high priority? IMHO, this bug sure seems to qualify...

tags: added: full-boot
mikewhatever (mikewhatever) wrote :

Recently, I had to remove 42 old kernels that accumulated over about 18 months on an old PC. It only took about 80 minutes. Clearly, old kernels should be auto-purged by default, with an option to keep them available for those that need it. As is, users have to manually run apt-get autoremove, resort to script, or configure unattended upgrades, none of which should ever be expected from regular users.

mikewhatever (mikewhatever) wrote :

Here is a good one: https://ubuntuforums.org/showthread.php?t=2374043&p=13697015#post13697015. The OP had, in total, 74 kernels, 65 headers and 93 extras = 232 packages that almost filled /, and gobbled up inodes. Don't know if it is a record, but it's close.

Displaying first 40 and last 40 comments. View all 127 comments or add a comment.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.