Karmic: Re-generation of grub.cfg takes long time.

Bug #425650 reported by moma on 2009-09-07
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
grub2 (Ubuntu)
Medium
Colin Watson

Bug Description

Hello,

This system is:
$ uname -a
Linux karmic64 2.6.31-9-generic #29-Ubuntu SMP Sun Aug 30 17:39:26 UTC 2009 x86_64 GNU/Linux

It takes quite long time to upgrade this system because the re-generation of GRUB2 (/boot/grub/grub.cfg) takes long time( ca. 10 - 12 minutes).

Here is the message log after running
$ sudo aptitude update && sudo aptitude upgrade

This part takes suspiciously long time:
...
Setting up memtest86+ (2.11-3ubuntu3) ...
Installing new version of config file /etc/grub.d/20_memtest86+ ...
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu 8.04 (8.04) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu 8.04 (8.04) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
done
...

It seems to scan the same partitions several times.
The scripts in /etc/grub.d/ are very fast if I run them maually. There should be no reason to waste time.
$ ls -l /etc/grub.d/*
--------------

System information:

$ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu karmic (development branch)
Release: 9.10
Codename: karmic
-------------

$ sudo fdisk -l
Disk /dev/sda: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0000b7d5

   Device Boot Start End Blocks Id System
/dev/sda1 1 5099 40957686 83 Linux
/dev/sda2 * 5100 10199 40965750 83 Linux
/dev/sda3 10200 15298 40957717+ 83 Linux
/dev/sda4 15299 30401 121314847+ 5 Extended
/dev/sda5 15299 20652 43005973+ 83 Linux
/dev/sda6 20653 26006 43005973+ 83 Linux
/dev/sda7 26007 26735 5855661 83 Linux
/dev/sda8 26736 27221 3903763+ 82 Linux swap / Solaris
/dev/sda9 27222 30401 25543318+ 83 Linux

Disk /dev/sdb: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000677e0

   Device Boot Start End Blocks Id System
/dev/sdb1 * 1 9726 78124063+ 83 Linux
/dev/sdb2 9727 19452 78124095 83 Linux
/dev/sdb3 19453 29178 78124095 83 Linux
/dev/sdb4 29179 38913 78196387+ 83 Linux

Disk /dev/sdc: 2034 MB, 2034237440 bytes
26 heads, 26 sectors/track, 5877 cylinders
Units = cylinders of 676 * 512 = 346112 bytes
Disk identifier: 0x04030201

   Device Boot Start End Blocks Id System
/dev/sdc1 * 60 5878 1966320 b W95 FAT32 (2 GB USB stick)
-------------

$ mount
/dev/sda2 on / type ext4 (rw,noatime,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,nosuid,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
securityfs on /sys/kernel/security type securityfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda7 on /tmp type ext3 (rw,noatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/moma/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=moma)
/dev/sdc1 on /media/DC73-EDF8 type vfat (rw,nosuid,nodev,uhelper=devkit,uid=1000,gid=1000,shortname=lower,dmask=0077,utf8=1,flush)
/dev/sdb4 on /media/sdb4_ type ext3 (rw,nosuid,nodev,uhelper=devkit)
-------------

moma (osmoma) wrote :
Colin Watson (cjwatson) wrote :

os-prober 1.31 will appear soon (not uploaded to Ubuntu yet - I'm waiting for it to land in Debian unstable first). When it does, could you please try with that and let me know if it's noticeably faster? It ought to be.

Regarding the duplicates, you can probably remove /etc/grub.d/30_otheros, which was mistakenly created by the installer (and won't be created in fresh installations as of Alpha 5).

affects: grub (Ubuntu) → grub2 (Ubuntu)
moma (osmoma) wrote :

Hello and thanks for your answer.

I will wait till the os-prober 1.31 package is available and will then do a new test.

This is the current "os-prober" package:

$ sudo apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 168
Maintainer: Ubuntu Installer Team <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.30ubuntu1
Filename: pool/main/o/os-prober/os-prober_1.30ubuntu1_amd64.deb
----

I also removed the "/etc/grub.d/30_otheros" script.

$ ls -l /etc/grub.d/
-rwxr-xr-x 1 root root 3175 2009-09-04 21:42 00_header
-rwxr-xr-x 1 root root 1154 2009-08-10 14:18 05_debian_theme
-rwxr-xr-x 1 root root 3557 2009-09-04 21:42 10_linux
-rwxr-xr-x 1 root root 476 2009-09-04 23:26 20_memtest86+
-rwxr-xr-x 1 root root 5245 2009-09-04 21:42 30_os-prober
-rwxr-xr-x 1 root root 84 2009-07-16 12:18 40_custom
-rw-r--r-- 1 root root 483 2009-07-16 12:18 README
----

Talk to you later.

moma (osmoma) wrote :

Re-hello,

Ok, new updates just arrived with os-prober v1.31 package.

a) First, upgrading an existing system. Here is the message log after running
$ sudo aptitude update && sudo aptitude upgrade

...
(hd0) /dev/sda
(hd1) /dev/sdb
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu 8.04 (8.04) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
done
--

Now the GRUB2 update took only a few seconds (maybe 5 - 6 seconds or so).

But some menu entries are missing.
The newly-generated /boot/grub/grub.cfg file contains only entries from this actual Ubuntu installation + rescue mode and memtest86. It does not include the various Linuxes from other partitions. Most notably the "Other operating systems:" section of grub.cfg is completly empty. See the attached grub.cfg.
-----

The version of os-prober is now:

$ sudo apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 180
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.31
Depends: libc6 (>= 2.4)
Filename: pool/main/o/os-prober/os-prober_1.31_amd64.deb
Size: 20784
MD5sum: 77a765fd961d3c972ef59b05d2572a61
...
--------

b) I also did a totally new 64bit Ubuntu Alpha5 installation.
The grub.cfg after first boot is exactly 100% same as in the upgrade step a).

And files in /etc/grub.d/ are:
$ ls -l /etc/grub.d/
-rwxr-xr-x 1 root root 2944 2009-09-01 23:53 00_header
-rwxr-xr-x 1 root root 1154 2009-09-01 23:41 05_debian_theme
-rwxr-xr-x 1 root root 3434 2009-09-01 23:53 10_linux
-rwxr-xr-x 1 root root 472 2009-06-24 18:04 20_memtest86+
-rwxr-xr-x 1 root root 5123 2009-09-01 23:53 30_os-prober
-rwxr-xr-x 1 root root 84 2009-09-01 23:53 40_custom
-rw-r--r-- 1 root root 483 2009-09-01 23:53 README
--------

The missing menu entries (the other partitions/Linuces) are not a big problem because I always maintain my own grub.cfg manually.

But maybe the "30_otheros" script is still required? Or maybe 30_os-prober script needs som fine-tuning?

See the attached grub.cfg.

Do not hesitate to ask me to do additional tests. I'll very happy to help. That's what I do.
--------

The missing entries for other Linux systems are due to bug 426061.
os-prober 1.32 is on its way to correct this.

moma (osmoma) wrote :

Re-hi,

The updates of today brought os-prober v1.32 which seems to work right.
It generated a new grub.cfg by scanning all the partitions once and in order, with correct result.

...
Running postinst hook script /usr/sbin/update-grub.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-10-generic
Found initrd image: /boot/initrd.img-2.6.31-10-generic
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu karmic (development branch) (9.10) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
done
----

$ apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 180
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.32
Depends: libc6 (>= 2.4)
Filename: pool/main/o/os-prober/os-prober_1.32_amd64.deb
Size: 20870
MD5sum: 0b861b394f23756f2f174f6453632416
----

Colin Watson (cjwatson) wrote :

On Wed, Sep 09, 2009 at 12:49:37PM -0000, moma wrote:
> The updates of today brought os-prober v1.32 which seems to work right.
> It generated a new grub.cfg by scanning all the partitions once and in order, with correct result.

OK, great. Was it still acceptably quick? If so, I think this bug can be
closed.

moma (osmoma) wrote :

Hallo,

Yes, it is acceptably quick and normal.
This machine has 11 partitions and totally 25 kernels, so it takes ca. 65 seconds to re-generate the entire grub.cfg. That is not bad at all. Very normal indeed.

Colin Watson (cjwatson) wrote :

Excellent - thanks for following up. Of course we may see if we can make it faster still (I have at least one idea that might help).

Changed in grub2 (Ubuntu):
assignee: nobody → Colin Watson (cjwatson)
status: New → Fix Released
ThyMythos (thymythos) wrote :

The problem still exists for me:

During "Generating grub.cfg..." the disk is under heavy load, braking out the whole system. The update takes approx. 20min.

System is:
$ uname -a
Linux mythos-e6500 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 9.10
Release: 9.10
Codename: karmic

$ sudo fdisk -l

Platte /dev/sda: 160.0 GByte, 160041885696 Byte
255 Köpfe, 63 Sektoren/Spuren, 19457 Zylinder
Einheiten = Zylinder von 16065 × 512 = 8225280 Bytes
Disk identifier: 0xc8000000

   Gerät boot. Anfang Ende Blöcke Id System
/dev/sda1 1 17 136521 de Dell Utility
/dev/sda2 18 279 2097152 7 HPFS/NTFS
Partition 2 endet nicht an einer Zylindergrenze.
/dev/sda3 * 279 5379 40970703+ 7 HPFS/NTFS
/dev/sda4 5380 19457 113081535 5 Erweiterte
/dev/sda5 5380 8619 26025268+ 83 Linux
/dev/sda6 8620 11231 20980858+ 83 Linux
/dev/sda7 11232 18880 61440561 83 Linux
/dev/sda8 18881 19457 4634721 82 Linux Swap / Solaris

$ mount
/dev/sda6 on / type ext4 (rw,relatime,errors=remount-ro)
proc on /proc type proc (rw)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
/dev/sda7 on /home type xfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/mythos/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=mythos)

ThyMythos (thymythos) wrote :

Still affects me

Changed in grub2 (Ubuntu):
status: Fix Released → Confirmed
ThyMythos (thymythos) wrote :

I think my problem could have been related to an 32-bit xfs partition on an 64-bit OS. I did a 'xfs_repair' and now the search time is much shorter.

Karl Hegbloom (karl.hegbloom) wrote :

I'm seeing it also. The commands that are taking so long are:

root 17586 0.0 0.0 4072 672 pts/7 S+ 11:27 0:00 | \_ /bin/sh -e /usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg 2.6.31-17-generic /boot/vmlinuz-2.6.31-17-generic
root 17636 76.4 0.1 17988 4672 pts/7 D+ 11:28 0:11 | \_ /usr/sbin/grub-probe -t fs /usr/share/grub/unicode.pf2

... running from within the update-grub from a postrm script, I believe. What is 'unicode.pf2' and why does grub-probe need to scan it?

Karl Hegbloom (karl.hegbloom) wrote :

This is untested, but I bet it solves the problem.

tags: added: patch
Changed in grub2 (Ubuntu):
importance: Undecided → Medium
Karl Hegbloom (karl.hegbloom) wrote :

I've tested that patch, and it works just fine. Please apply it to the package before you release Lucid Lynx or you'll be hearing about it a lot more.

Colin Watson (cjwatson) wrote :

Sorry, I think this patch is incorrect as it stands. The point of using grub-probe is to ensure that GRUB will be able to read it at boot time. 'test -r' would not achieve this - it would only check for file permissions, which are irrelevant here (we've already done 'test -e').

grub-probe should not take a long time, and if it is doing so then *that* is what should be fixed. That particular call is not in an inner loop or anything - indeed I think it's only called once per update-grub run, and there's no intrinsic reason it should be a bottleneck.

Colin Watson (cjwatson) wrote :

Oh, and I don't normally find "Please <do X> or you'll be hearing about it a lot more" particularly persuasive, by the way - I prefer solid technical arguments!

Colin Watson (cjwatson) wrote :

I'm currently seeking review for http://lists.gnu.org/archive/html/grub-devel/2010-03/msg00008.html; if I don't get serious objections upstream (and so far people seem to like it, although it's a bit risky for 1.98 which is coming out very soon), I'll apply it to Lucid.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package grub2 - 1.98~20100128-1ubuntu4

---------------
grub2 (1.98~20100128-1ubuntu4) lucid; urgency=low

  [ Colin Watson ]
  * Update DM-RAID probing patch, removing some configure.ac cruft.
  * Add patch from Colin King to shave eight bytes off the
    pre-partition-table part of boot.img.
  * Don't display "GRUB loading" unless Shift is held down.
  * Use light-gray as the default background for highlighted menu entries;
    white is not safe because some systems interpret the top bit as blink
    (thanks, proski; LP: #527334).
  * Adjust versions of grub-doc and grub-legacy-doc conflicts to tolerate
    our backport of the grub-doc split (LP: #493968).
  * Optimise hostdisk device handling, substantially speeding up grub-probe
    filesystem reads (LP: #425650).
 -- Colin Watson <email address hidden> Wed, 03 Mar 2010 20:33:03 +0000

Changed in grub2 (Ubuntu):
status: Confirmed → Fix Released
Lex Ross (lross) wrote :

I have the same problem on 11.04 (Natty) now using 64-bit kernel 2.6.38-8-generic and grub-probe version 1.99~rc1-13ubuntu3
The reason is mystery to me as the update log come clean like this,

run-parts: executing /etc/kernel/postinst.d/initramfs-tools 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/nvidia-common 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.38-10-generic
Found initrd image: /boot/initrd.img-2.6.38-10-generic
Found linux image: /boot/vmlinuz-2.6.38-8-generic
Found initrd image: /boot/initrd.img-2.6.38-8-generic
Found memtest86+ image: /memtest86+.bin
done

It's just a long wait (10-12 minutes) after "Generating grub.cfg ..." message.
In fact, it takes so long that I had aborted my first installation on Natty thinking it had failed at this point.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers