free space check doesn't take into account kernel upgrades

Bug #1640645 reported by Manna Akpan
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
Fix Released
Medium
Brian Murray
Xenial
Fix Released
High
Brian Murray

Bug Description

[Test Case]
If you don't have a separate /boot partition you can create a file which will be loopback mounted as boot and then test having a full /boot.
1) Install xenial hwe backport kernel
2) dd if=/dev/zero of=fake-boot bs=1024 count=204800
3) mkfs -t ext3 fake-boot
4) copy your real /boot somewhere
5) sudo mount -t ext3 fake-boot /boot
6) copy the files from your old /boot into the /boot
7) fill /boot up some more e.g. fallocate -l 50M filler.txt
8) upgrade from Trusty to Xenial (do-release-upgrader)

With the version of the release-upgrader in the archive you'll receive an error regarding "gzip: stdout: No space left on device... mkinitramfs failure cpio 141 gzip 1".

With the version of the release-upgrader from -proposed you'll receive an error notification that there is "Not enough free disk space" and the upgrade will not proceed.

[Regression Potential]
There is little chance of regression as we are just checking to see if linux-image or linux-image-debug packages are marked for upgrade.

Could not install 'linux-image-4.4.0-45-generic'

ProblemType: BugDistroRelease: Ubuntu 16.04
Package: ubuntu-release-upgrader-core 1:16.04.17
ProcVersionSignature: Ubuntu 4.4.0-45.66~14.04.1-generic 4.4.21
Uname: Linux 4.4.0-45-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.1
Architecture: amd64
CrashDB: ubuntu
CurrentDesktop: Unity
Date: Wed Nov 9 20:11:19 2016
InstallationDate: Installed on 2015-05-24 (535 days ago)
InstallationMedia: Ubuntu 14.04.2 LTS "Trusty Tahr" - Release amd64 (20150218.1)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_CA:en
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_CA.UTF-8
 SHELL=/bin/bashSourcePackage: ubuntu-release-upgrader
Symptom: release-upgrade
UpgradeStatus: Upgraded to xenial on 2016-11-10 (0 days ago)
VarLogDistupgradeAptclonesystemstate.tar.gz: Error: command ['pkexec', 'cat', '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'] failed with exit code 127: Error creating textual authentication agent: Error opening current controlling terminal for the process (`/dev/tty'): No such device or address
VarLogDistupgradeTermlog:

Revision history for this message
Manna Akpan (makpan) wrote :
tags: added: trusty2xenial
Revision history for this message
Brian Murray (brian-murray) wrote :

You seem to have run out of disk space, likely in /boot if that is a separate partition.

Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-38-generic /boot/vmlinuz-4.4.0-38-generic
run-parts: executing /etc/kernel/postinst.d/dkms 4.4.0-38-generic /boot/vmlinuz-4.4.0-38-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-38-generic /boot/vmlinuz-4.4.0-38-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-38-generic
W: plymouth module (/usr/lib/x86_64-linux-gnu/plymouth//.so) missing, skipping that theme.

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-4.4.0-38-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1

Please try removing some old kernels and running 'sudo apt-get -f install'. Thanks!

Revision history for this message
Brian Murray (brian-murray) wrote :

Looking further in the log files multiple linux-image-4.4* packages are marked for upgrade e.g. "Unpacking linux-image-4.4.0-47-generic (4.4.0-47.68) over (4.4.0-47.68~14.04.1)". However, packages marked for upgrade and not included when calculating the amount of free space needed in boot so we have the following:

2016-11-09 18:57:31,611 DEBUG dir '/boot' needs '0' of '<DistUpgrade.DistUpgradeCache.MyCache.checkFreeSpace.<locals>.FreeSpace object at 0x7fe166fe1be0>' ( 51766272.000000)

It's not clear to me if any free space is actually required in boot to upgrade the kernel.

Revision history for this message
Brian Murray (brian-murray) wrote :

Testing an upgrade from T to X with the the -lts-xenial stack installed we can find out the following using ipdb in checkFreeSpace.

ipdb> pkg.name
'linux-image-4.4.0-47-generic'
ipdb> pkg.installed
<Version: package:'linux-image-4.4.0-47-generic' version:'4.4.0-47.68~14.04.1'>
ipdb> pkg.marked_upgrade
True

So it seems like we should check to see if the pkg is marked_upgrade.

summary: - subprocess installed post-installation script returned error exit status
- 2
+ free space check doesn't take into account kernel upgrades
Changed in ubuntu-release-upgrader (Ubuntu):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Brian Murray (brian-murray)
Revision history for this message
Brian Murray (brian-murray) wrote :

I fixed this in zesty but forgot to reference the bug.

ubuntu-release-upgrader (1:17.04.4) zesty; urgency=medium

  * DistUpgrade/DistUpgradeCache.py: when checking for free space in boot also
    check for kernels being upgraded not just installed.

Revision history for this message
Brian Murray (brian-murray) wrote :

This only needs SRU'ing to 16.04 since 14.04 is the only release where people might be running the same kernel as in the release to which they are upgrading.

Changed in ubuntu-release-upgrader (Ubuntu):
status: In Progress → Fix Released
Changed in ubuntu-release-upgrader (Ubuntu Xenial):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
description: updated
description: updated
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Manna, or anyone else affected,

Accepted ubuntu-release-upgrader into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:16.04.20 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in ubuntu-release-upgrader (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Testing the package from -proposed I ran into the following error, which is the desired effect:

Not enough free disk space

The upgrade has aborted. The upgrade needs a total of 69.8 M free
space on disk '/boot'. Please free at least an additional 41.8 M of
disk space on '/boot'. Empty your trash and remove temporary packages
of former installations using 'sudo apt-get clean'.

bdmurray@upgrade-trusty-amd64:~$ cat /tmp/ubuntu-release-upgrader-gtif11_8/DistUpgradeVersion.py
VERSION = '16.04.20'

Setting to verification-done.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:16.04.20

---------------
ubuntu-release-upgrader (1:16.04.20) xenial; urgency=medium

  * DistUpgrade/DistUpgradeCache.py: when checking for free space in boot also
    check for kernels being upgraded not just installed. (LP: #1640645)

 -- Brian Murray <email address hidden> Fri, 02 Dec 2016 10:54:40 -0800

Changed in ubuntu-release-upgrader (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers