do-release-upgrade should check for free space in the actual paths where it's needed

Bug #442456 reported by ariel cornejo
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
update-manager (Ubuntu)
Incomplete
Low
Michael Vogt

Bug Description

Binary package hint: update-manager

When running 'do-release-upgrade -d', it complains about not enough space in /var (978 MB needed). I didn't want to repartition so by suggestion of NoelJB and several others (thanks), I symlinked /var/cache/apt and /var/tmp to new directories (root filesystem, 2.2 GB available, see attachment). The upgrade failed with the same complaint about needing more space in '/var'.

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
NonfreeKernelModules: nvidia
Package: update-manager-core 1:0.111.9
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: update-manager
Uname: Linux 2.6.31-020631-generic i686
UnreportableReason: This is not a genuine Ubuntu package

Revision history for this message
ariel cornejo (arielco) wrote :
Revision history for this message
Noel J. Bergman (noeljb) wrote :

This appears to be Bug 106804 again. That one is marked as fixed. Haven't looked to see why it is broken again.

Noel J. Bergman (noeljb)
tags: added: regression-potential
Revision history for this message
Noel J. Bergman (noeljb) wrote :

The patch from two years ago is in http://bazaar.launchpad.net/~<email address hidden>?start_revid=michael.vogt%40ubuntu.com-20070424111329-d5otfhnrwzlqamjc

The code is now in /usr/share/pyshared/DistUpgrade/DistUpgradeCache.py. There are three (3) loops, starting at 992 (figure out what free space we have), 1040 (figure out what we need), 1054 (figure out what failed). It seems that we have two entries in the first for /var:

        for d in ["/","/usr","/var","/boot", archivedir, aufs_rw_dir, "/home"]

Do we need the explicit /var entry? The second loop does not declare any requirement for /var, but the 3rd will check.

Changed in update-manager (Ubuntu):
assignee: nobody → Michael Vogt (mvo)
status: New → Confirmed
Revision history for this message
Noel J. Bergman (noeljb) wrote :

Believe that this may do the trick.

Revision history for this message
Wesley Schwengle (wesleys) wrote :

I agree with Noel that /var doesn't need to be checked for free diskspace for the dist-upgrade.

We've given ariel the following patch, where we don't check /var/ but /var/apt/archive instead. The upgrade process continues as expected. The patch is checking archivedir twice, so I guess it is safe to say that /var can be removed from the check and we don't have to explicitly check /var/cache/apt.

--- DistUpgradeCache.py.orig 2009-10-04 20:55:51.000000000 +0200
+++ DistUpgradeCache.py 2009-10-04 20:54:56.000000000 +0200
 -989,7 +989,7 @@
             if not os.path.exists(aufs_rw_dir):
                 os.makedirs(aufs_rw_dir)
         logging.debug("cache aufs_rw_dir: %s" % aufs_rw_dir)
- for d in ["/","/usr","/var","/boot", archivedir, aufs_rw_dir, "/home"]:
+ for d in ["/","/usr","/var/cache/apt","/boot", archivedir, aufs_rw_dir, "/home"]:
             d = os.path.realpath(d)
             fs_id = make_fs_id(d)
             st = os.statvfs(d)

Revision history for this message
ariel cornejo (arielco) wrote :

I tried Noel's patch for Bug 442456 and it works for me:

--- DistUpgradeCache.py~ 2009-09-29 15:02:52.000000000 -0400
+++ DistUpgradeCache.py 2009-10-04 15:53:04.000000000 -0400
@@ -989,7 +989,7 @@
             if not os.path.exists(aufs_rw_dir):
                 os.makedirs(aufs_rw_dir)
         logging.debug("cache aufs_rw_dir: %s" % aufs_rw_dir)
- for d in ["/","/usr","/var","/boot", archivedir, aufs_rw_dir, "/home"]:
+ for d in ["/","/usr","/boot", archivedir, aufs_rw_dir, "/home"]:
             d = os.path.realpath(d)
             fs_id = make_fs_id(d)
             st = os.statvfs(d)

Thanks!

Revision history for this message
Michael Vogt (mvo) wrote :

Thanks for your bugreport and the patch.

I would like to see the /var/log/dist-upgrade/main.log file from the failure if that is still available (it should be saved in one of the subdirs).

Changed in update-manager (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
ariel cornejo (arielco) wrote :

Here it is. The upgrade failed anyway (dpkg --configure segfaulted and my system was left unusable until today), but it did finish the download.

Revision history for this message
Michael Vogt (mvo) wrote :

Thanks! That is the one with /var removed in the the source, right? I wonder if you still have a copy of the one where /var was still in and when it refused to upgrade because of space problems.

Revision history for this message
ariel cornejo (arielco) wrote :

> That is the one with /var removed in the the source, right?
Nope - one such upgrade is running now. When I get back home I'll post it too.

> I wonder if you still have a copy of the one where /var was still in and when it refused to upgrade because of space problems.
It's the one I posted:
    2009-10-04 12:05:13,506 ERROR Not enough free space: ['/var']

Steve Langasek (vorlon)
Changed in update-manager (Ubuntu):
importance: Undecided → Low
Revision history for this message
Pavek (avezzandrog) wrote :

I can't upgrade for the same reason, from 9.04 to 9.10.

While browsing files, watching / folder, it says at the bottom of the window 827Mb of free space, however I have more than 300 gigs of it, checked with baobab.

Revision history for this message
Pavek (avezzandrog) wrote :

Even after patching, still no luck :(

Revision history for this message
ariel cornejo (arielco) wrote :

Pavek, please post the output of:
    df -h /var /var/tmp var/cache/apt/archives
and then:
    sudo do-release-upgrade -d
The 'do-release-upgrade' command should be run from the same directory where you applied the patch (/tmp/something/)

Revision history for this message
chatmoa (jpherbien) wrote :
Download full text (3.3 KiB)

Same pb for me:

/var/log/dist-upgrade/main.log
...
2009-11-17 11:21:56,873 DEBUG cache aufs_rw_dir: /tmp/
2009-11-17 11:21:56,874 DEBUG Free space on /: 1639116800
2009-11-17 11:21:56,874 DEBUG Free space on /usr: 1639116800
2009-11-17 11:21:56,874 DEBUG Dir /var mounted on /
2009-11-17 11:21:56,874 DEBUG Dir /boot mounted on /
2009-11-17 11:21:56,875 DEBUG Dir /u/archives mounted on /usr
2009-11-17 11:21:56,875 DEBUG Dir /tmp mounted on /
2009-11-17 11:21:56,875 DEBUG Free space on /home: 1896259584
2009-11-17 11:21:56,876 DEBUG Dir /tmp mounted on /
2009-11-17 11:21:56,876 DEBUG fs_free contains: '{'/var': <DistUpgradeCache.FreeSpace object at 0x95d2f4c>, '/tmp': <DistUpgradeCache.FreeSpace object at 0x95d2f4c>, '/usr': <DistUpgradeCache.FreeSpace object at 0x95d2aec>, '/boot': <DistUpgradeCache.FreeSpace object at 0x95d2f4c>, '/home': <DistUpgradeCache.FreeSpace object at 0x95d2e0c>, '/': <DistUpgradeCache.FreeSpace object at 0x95d2f4c>, '/u/archives': <DistUpgradeCache.FreeSpace object at 0x95d2aec>}'
2009-11-17 11:21:56,975 DEBUG linux-image-2.6.31-15-generic (new-install) added with 15728640 to boot space
2009-11-17 11:21:57,712 DEBUG dir '/u/archives' needs '1827157578.0' of '<DistUpgradeCache.FreeSpace object at 0x95d2aec>' (1639116800.000000)
2009-11-17 11:21:57,712 DEBUG dir '/usr' needs '1030918144.0' of '<DistUpgradeCache.FreeSpace object at 0x95d2aec>' (-188040778.000000)
2009-11-17 11:21:57,712 DEBUG dir '/usr' needs '52428800' of '<DistUpgradeCache.FreeSpace object at 0x95d2aec>' (-1218958922.000000)
2009-11-17 11:21:57,712 DEBUG dir '/boot' needs '15728640' of '<DistUpgradeCache.FreeSpace object at 0x95d2f4c>' (1639116800.000000)
2009-11-17 11:21:57,712 DEBUG dir '/tmp' needs '5242880' of '<DistUpgradeCache.FreeSpace object at 0x95d2f4c>' (1623388160.000000)
2009-11-17 11:21:57,712 DEBUG dir '/' needs '10485760' of '<DistUpgradeCache.FreeSpace object at 0x95d2f4c>' (1618145280.000000)
2009-11-17 11:21:57,713 DEBUG dir '/tmp' needs '0.0' of '<DistUpgradeCache.FreeSpace object at 0x95d2f4c>' (1607659520.000000)
2009-11-17 11:21:57,713 ERROR Not enough free space: ['/u']
2009-11-17 11:22:02,591 DEBUG abort called
2009-11-17 11:22:02,594 DEBUG openCache()
2009-11-17 11:22:06,300 DEBUG /openCache(), new cache size 27061
2009-11-17 11:22:06,301 DEBUG enabling apt cron job

moi@moi:/u/archives> ls -la /var/cache/apt
total 25936
drwxr-xr-x 3 root root 4096 2009-11-17 11:24 .
drwxr-xr-x 21 root root 4096 2009-08-26 11:03 ..
drwxr-xr-x 2 root root 4096 2007-10-23 13:56 apt-file
lrwxrwxrwx 1 root root 11 2009-04-24 10:00 archives -> /u/archives
-rw-r--r-- 1 root root 13428310 2009-11-17 11:25 pkgcache.bin
-rw-r--r-- 1 root root 13125376 2009-11-17 11:24 srcpkgcache.bin

df -h gives :

Sys. de fich. Tail. Occ. Disp. %Occ. Monté sur
/dev/sda2 9,1G 7,1G 1,6G 82% /
tmpfs 1006M 0 1006M 0% /lib/init/rw
varrun 1006M 236K 1006M 1% /var/run
varlock 1006M 0 1006M 0% /var/lock
udev 1006M 152K 1006M 1% /dev
tmpfs 1006M 172K 1006M 1% /dev/shm
/dev/sda5 5,5G 3,...

Read more...

Revision history for this message
chatmoa (jpherbien) wrote :

The error is here :
...
2009-11-17 11:49:05,012 DEBUG Dir /u/archives mounted on /usr
...

the right is :
/u/archives mounted on /u

Revision history for this message
chatmoa (jpherbien) wrote :

this fixes the pb :

DistUpgradeCache.py
line 976
add : +"/"

...
   if d.startswith(mount_point+"/"):
       return mount_point
...

tags: added: regression-release
removed: regression-potential
Revision history for this message
bsh (bsh) wrote :

still not working...
tried a "do-release-upgrade -d" from 10.04 to 12.04, the / filesystem is 6.0G (5.7), of which 3.6G is used. the release upgrade failed, because it ran out of space halfways, rendering the system unusable, and with a lot of leftover mess.
the script should REALLY check for enough space for it to complete.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.