Not enough disk space for kernel security update on /boot

[Patrick Baenziger]

Hi all

I am using Ubuntu 12.10 and have just received the newest update notification from the update-manager marked as security updates. (This is NOT a release upgrade, but the regular security update)

The update-manager tells me now that there is insufficient space on /boot to install these updates.
The problem obviously is that too many kernels are installed: 3.5.0-17, 25, 26, 27, 28 and 30

Which leaves 27MB space:
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 228M 190M 27M 88% /boot

Update Manager reports that it needs at least 33.2MB space for this update:

"The upgrade needs a total of 33.2 M free space on disk '/boot'. Please free at least an additional 5,520 k of disk space on '/boot'. Empty your trash and remove temporary packages of former installations using 'sudo apt-get clean'."

The hint in the message does not help for /boot.
While I can fix this myself, I believe that this is not acceptable for an average end user to research and fix.
Update-Manager should at least offer the option to remove old kernels (which it installed itself by security updates) in order for the security updates to proceed.

I found an answer on AskUbuntu.com (http://askubuntu.com/questions/142926/cant-upgrade-due-to-low-disk-space-on-boot) which will help fix the issue.

TL;DR:

-- 1 -- Release: 12.10
-- 2 -- Installed Version of update-manager: 1:0.174.4
-- 3 -- Expected:
Security update should be installed.
-- 4 -- Happened:
Failed because of insufficient disk space on /boot
(Too many old kernels previously installed and not removed by update-manager)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[tankdriver (stoneraider-deactivatedaccount)]

Same Problem here on 13.10. Old kernels should be removed automatically when this happens.

[Stuart Bishop (stub)]

Per tankdriver's comment, if old kernels cannot be automatically and safely removed somehow then Software Updater's automatic security updates will start failing after three or four kernel updates.

[Matt Hanyok (matthew-hanyok)]

This has continued in 14.04 LTS. If the release is intended for long term support, the /boot partition should probably be larger when the OS is installed and Ubuntu creates the partitions. This is only a temp fix, though.

I also think there should be a menu that pops up for removal of older kernel versions, as that's really the only long-term fix.

[Pablo W (pablowains)]

This bug also affects me in 14.04 LTS. I have seen the Askubuntu answer reported above by Patrick Baenziger. Not being an expert, the idea of messing around with old kernels is not something I feel confident enough to do.

[Franck (alci)]

This seems pretty bad, and mainly unmanageable by most average users.
A lot of question are generated, like these:
http://askubuntu.com/questions/342820/trying-to-update-but-not-enough-space-on-boot-answer-here
http://askubuntu.com/questions/298487/not-enough-free-disk-space-when-upgrading
http://askubuntu.com/questions/142926/cant-upgrade-due-to-low-disk-space-on-boot
http://ubuntuforums.org/showthread.php?t=2142894

and the most accepted answer is to blindly copy and paste this magic incantation:

dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

A better solution has to be found, I can't explain to my Daddy why "there is not enough disk space" :-)

14.04.01 LTS (!!)

[Nathaniel Homier (mechamechanism)]

I guess this bug must not be happening to very many people if it's not fixed a year later.

[Matt Hanyok (matthew-hanyok)]

Just got the error from update-manager again today. I'm still on 14.04.

I can go into Synaptic and manually remove individual older installed versions of the kernel, and the line that Franck posted will go and remove all old versions of the kernel that are installed. Either one frees up space.

Is it possible to have update manager display a list of installed kernels and give the option of "uninstall all" or "uninstall selected"?

[Kosmo Tatalias (kosmotat)]

The easiest fix for me when I first encountered the not enough /boot space message was to create a boot disk and completely replace the existing OS, then restore files from a backup drive. A couple of upgrades later, same problem with not enough /boot space. My poor vision makes me uncomfortable having to type in long strings which affect OS files, and I would greatly appreciate a fix as suggested by Matt Hanyok above.

[manfreed (manfreed)]

This is still an issue in 14.10

This error message is too general, it doesn't give us a specific solution:

"The upgrade needs a total of 86,4 M free space on disk '/boot'. Please free at least an additional 32,2 M of disk space on '/boot'. Empty your trash and remove temporary packages of former installations using 'sudo apt-get clean'."

The minimum is to offser solution based on the partition. For /boot the proper solution would be to tell the users to remove old kernels. They wouldn't be able to do themselves without help or proper knowledge, but at least the message wouldn't be confusing.

The best solution I can think of:

If the problematic partition is /boot, and there are more than one installed kernels:

Tell the users that their boot partition is filled with older kernels which are mostly of no use. Tell them that they are safe to remove, and offer them to remove manually. Let the user choose which kernel they want to keep tough. A list of kernels with checkboxes would be fine, just check the old kernels by default and don't let the users to select eveything. If they try, tell them why it isn't a good idea.

Another problem is that apt wouldn't even complain about not having enough space, it just tries to do it's job and fails.

[Matt Hanyok (matthew-hanyok)]

Just got an error again about this today.

[Jay Cassano (cassano-jay)]

If it helps, I'm pretty sure this bug only happens on computers with full disk encryption (or possibly home folder encryption - not sure). Has been going on for over a year for me and is pretty ugly.

[Teemu Leisti (teemu-leisti)]

Yep, it affects me too, at least a couple of times a year. I'm using home folder encryption only, not full disk encryption. When installing Xubuntu, home-folder encryption is given as an option, so this bug will affect those "ordinary" users who chose that option, not just us nerds. So, I also think that when the "insufficient space on /boot" dialog is shown, there should be an option for the user to have old kernel versions removed automatically, because it's too difficult for the ordinary user to remove them using via the command line.

[Devvyn Murphy (devvyn)]

For reference, the official help page that pertains to fixing a full boot volume due to old kernel images is [https://help.ubuntu.com/community/RecoverLostDiskSpace#line-160], which recommends using Synaptic Package Manager and the command line together. I mention it because it seems to be a more appropriate solution than the popular incantations, and it would be nice to steer users toward a GUI solution when the aforementioned failure message arises in Update Manager, during an attempted kernel upgrade.

Also, for reference, I ran into this issue pretty quickly after installing Ubuntu 14.04 with installer defaults. At no point was I guided toward an appropriate boot volume size, so I feel this is likely to affect a lot of users, whether or not they find this old bug report and voice their dismay.

[Richard (ric4ard)]

Affecting me on 14.04 with full disk encryption.
As per the comments above, it's not complicated to resolve if you are technical but I wouldn't want to recommend Ubuntu to a non technical user while this could occur.

[sokolov (daniel-sokolov)]

Problem still persists in 15.04.

Also, it seems that the partition is simply too small. So this should be changed in the installation routine, too. I only have two kernel images there - the current one and the previous one. And I can't update because there is insufficient space.

[Teemu Leisti (teemu-leisti)]

Good points above. So, in addition to the update-manager offering to remove earlier kernel versions to free up space when necessary, the standard installation option should make the boot partition bigger.

[Matt Hanyok (matthew-hanyok)]

Still affecting 15.10 - haven't tried the 16.04 LTS betas yet.

[Matt Hanyok (matthew-hanyok)]

This affects the upgrade tool as well - if the user gets the prompt to do the update to 16.04 and clicks "upgrade", it downloads the update tool, starts to check and see if it can do the upgrade, and then fails with the same "not enough disk space" error.

It also doesn't offer an easy way to correct the issue and the continue running the upgrade, so the user is left with an error box that can only be closed and no easy way to re-start the update after fixing the problem (assuming they can figure out how to fix the problem).

The last few times I've seen this issue, it seems that apt-get autoremove actually correctly identifies the old installed kernel files and removes them properly (it was inconsistent before as to whether autoremove would work) - perhaps simply offering the user an option to 'clean up old system files' and then run autoremove would be sufficient?

[Matt Hanyok (matthew-hanyok)]

And, as of today's attempt at installing some updates, it is still happening in 16.04.

[Rob Hills (rhills)]

Agree with the above. There should be a simple, foolproof-as-possible way to manage boot device free space. I am currently trying to rescue my wife's computer from an incomplete 14.04LTS -> 16.04LTS upgrade that has failed near the end because it ran out of space on /boot. THIS SHOULD NOT HAPPEN!
Maybe I should have known to check for adequate free space on /boot before doing an upgrade, but if I should have known that, so should the upgrader and it should have checked and warned me BEFORE STARTING the upgrade!

Ideally the upgrade process should offer the option of clearing out old kernels (there were only 2 there apart from the one in use) before proceeding.

[Robert (birmingham-spider)]

This one has just bitten me, and necessitated research to fix it. The suggestion offered by the updater, to run "sudo apt-get clean", not only does not address the issue (because the old kernels are left in place), but also is user-hostile. Why not ask the user if they want to clean up from past upgrades (of all kinds), and then just get on with everything necessary?

[Matt Hanyok (matthew-hanyok)]

Still happening. I noticed that running apt-get autoremove usually clears up enough old kernel versions to fix this, but it didn't work this time.

Surely the updater can be configured to run autoremove and prompt the user yes/no before attempting to install updates? That may fix the issue in a large number of cases; then it would just be a matter of figuring out what is marking some kernel versions as still in use so they're not caught by autoremove.

Also, a larger /boot would still be a good idea.

[Mitch Singler (msingler)]

I am affected by this bug/issue for any updates requiring space in /boot not just security updates. I only have one working kernel and a couple of partially configured kernels as a result of my efforts to free up space by using the apt-get purge command. Here is my current state:

_________________________________________________
mitch@Mitch-desktop:~$ dpkg -l | grep "linux-image"
ii linux-image-4.4.0-36-generic 4.4.0-36.55 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-38-generic 4.4.0-38.57 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
iF linux-image-4.4.0-43-generic 4.4.0-43.63 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-36-generic 4.4.0-36.55 amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
iF linux-image-extra-4.4.0-38-generic 4.4.0-38.57 amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
iU linux-image-extra-4.4.0-43-generic 4.4.0-43.63 amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
iU linux-image-generic 4.4.0.43.45 amd64 Generic Linux kernel image

________________________________

Is my next option to boot into a partition manager and increase the size of the /boot partition?

[Not Martin Wimpress (not-martinwimpress)]

My parents got this error message on their LM18 box. They are not technical people; that's why they have Mint installed. When I heard about this, my jaw literally dropped.

Can someone tell me what the point is to store old and "insecure" kernels? Considering the frequency with which Ubuntu has been pushing out security kernels this year, you would think maybe the last one (or two maximum) is kept in case of a corrupt condition -- not all of them -- unnecessarily clogging their /boot partition.

This needs to be worked on. Immediately.

[Not Martin Wimpress (not-martinwimpress)]

Actually I think this is the parent bug for anyone watching: https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1357093

LM18 has a really poor implementation of "unattended-upgrades" and isn't properly configured by default and (as a part of the package) isn't running a pseudo-apt-autoremove command built into the latest version.

If any of you are reading this and are on 16.04, is "unattended-upgrades" installed by default?

[Yngvar Kristiansen (yngvark)]

This bug needs to be fixed. Why the hell do I need to know about this shit to use Ubuntu normally?

[Not Martin Wimpress (not-martinwimpress)]

Apparently, according to this page: https://help.ubuntu.com/community/RemoveOldKernels -- you're ultimately responsible for keeping /boot clean because there's no "automatic" mechanism by which old kernels are purged. I've found in my Lubuntu 16.04 installation that the easiest way of doing this is simply "sudo apt autoremove" but in other distros like LM, I would simulate a run first by doing a "sudo apt-get -s autoremove" to ensure nothing odd will get removed (such as VLC if xplayer is removed as VLC will act as an orphan dependency in that situation.) Many "unneeded dependency" packages can get marked as "auto removable" -- not just old kernels.

Btw, that Ubuntu article mentions autoremove not "purging" the old kernels...in my own testing on Lubuntu 16.04, the old kernels were indeed marked as "auto removable"...I simply did "sudo apt autoremove" and that freed up /boot to 25% used (a "freshly-normal" size with LVM), so I'm assuming that was an error (or someone else can correct me here.) The "unattended-upgrades" package was NOT installed by default...that's apparently another error in the article, as it says, "The unattended-upgrades package, included with the default install of all Ubuntu flavors..."

If you're on LM, then do the byobu -> "purge old kernels" method described here: https://forums.linuxmint.com/viewtopic.php?p=1242953#p1242953 -- for whatever reason, the devs there don't mark old kernels as "auto removable" so you need to jump these extra hoops on Mint.

Personally, I think a clear warning message should appear in the Ubiquity installer when choosing to select LVM encryption about how /boot needs to be *manually* cleaned after 3-4 kernel installs. That shouldn't be too hard to do right? It seems like there's a new kernel every 2 weeks these days -- which only compounds the problem.