update-grub never runs because ubiquity crashed with InstallStepError in configure_bootloader()

[David Favor]

Binary package hint: ubiquity

I'm filing this bug report because this is occurring in today's daily snapshot (2009-08-27).

This failure causes update-grub to be missed, resulting in a bricked machine.

There is no /var/log/installer directory. Neither faillog or dpkg.log show any useful data.

I've setup to install from USB now, so I can test .iso files every day.

If there's a packaging patch someone desires me to try, then do a reinstall, I can.

I'll leave this machine as an iso test machine till the problem is resolved.

ProblemType: Crash
Architecture: i386
Date: Thu Aug 27 13:07:04 2009
DistroRelease: Ubuntu 9.10
ExecutablePath: /usr/lib/ubiquity/bin/ubiquity
InterpreterPath: /usr/bin/python2.6
LiveMediaBuild: Ubuntu 9.10 "Karmic Koala" - Alpha i386 (20090827)
Package: ubiquity 1.99.12
ProcCmdline: /usr/bin/python /usr/lib/ubiquity/bin/ubiquity gtk_ui
ProcEnviron: Error: [Errno 13] Permission denied: '/proc/4785/environ'
ProcVersionSignature: Ubuntu 2.6.31-7.27-generic
PythonArgs: ['/usr/lib/ubiquity/bin/ubiquity', 'gtk_ui']
SourcePackage: ubiquity
Title: ubiquity crashed with InstallStepError in configure_bootloader()
Uname: Linux 2.6.31-7-generic i686
UserGroups:

[David Favor (davidfavor)]

I tried this... which failed...

mount /dev/sda1 /mnt -o rw
mount --bind /dev /mnt/dev -o rw
mount --bind /proc /mnt/proc -o rw
chroot /mnt
apt-get install linux-image-2.6.31-7-generic --reinstall

This reinstall worked with no errors and machine is still bricked - black screen on boot.

[Colin Watson (cjwatson)]

Unfortunately the "UbiquitySyslog.gz" attachment does not appear to be from a real run of ubiquity - it just looks like an ordinary system log. Could you try this again and manually attach /var/log/installer/syslog from the running live system to this bug, please?

[Colin Watson (cjwatson)]

Sorry, I misspoke: please attach /var/log/syslog from the running live system. Make sure to do this *before rebooting*.

[David Favor (davidfavor)]

Looks like install.py is dying.

Attached is the installer debug log.

[David Favor (davidfavor)]

I'll leave this machine in this state for a bit.

If someone likes me to gather more data or try to run any commands manually to recover, post the exact commands here.

[David Favor (davidfavor)]

After many experiments here's the cleanest way to recover from this to a bootable system.

mount /dev/sda1 /mnt -o rw
mount --bind /dev /mnt/dev
chroot /mnt
apt-get update
apt-install grub-pc

Just follow instructions and reboot.

[David Favor (davidfavor)]

I've begun to consider this may be a problem with the installer recognising disks correctly.

I'm installing on a 2TB WD20EADS-00R disk.

Attached is the fdisk -l which appears to be setting this disk up as a GPT (EFI/Apple bootable) disk.

Maybe because I have a Mac sitting next to the machine I'm installing? :-)

I'm by no means an expert and seems like I had to setup this disk type recently when installing MacOS on an external Firewire 800 drive.

Maybe this problem relates to new (large multi terrabyte) hard drives.

[David Favor (davidfavor)]

This bug does not appear in 9.04 as I can install it just fine and appears to specifically relate to grub 2 + large drives.

I've recreated this bug manually installing on small partitions, on a single partition on one drive with swap on another and even on a pristine drive cleaned by way of...

dd if=/dev/zero of=/dev/sda bs=512 count=1

In all cases the same problem occurs.

[David Favor (davidfavor)]

It appears this problem is queued to be fixed in Alpha 6. Guess I try Fedora.

https://bugs.launchpad.net/ubuntu/+source/partman-auto/+bug/417724 covers what appears to be the fix.

[David Favor (davidfavor)]

Partial fix to create a bootable system is to create a GPT friendly partition map. The explanation for this is long and circuitous. Google parted + GPT + howto for details.

Here's one parted map which allows GPT disks (> 2Tb) to be used smoothly, meaning grub without blocklists.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488111 for some details about blocklists.

root@home1:~# parted
GNU Parted 1.8.8.1.159-1e0e
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: ATA WDC WD20EADS-00R (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
 1 17.4kB 1018kB 1000kB bios_grub
 2 1018kB 9262MB 9261MB linux-swap(new)
 3 9262MB 29.3GB 20.0GB ext4

(parted)

Please note... There still a bug in latest development install which skips running update-grub when new kernels are installed. This still exists in today's linux-image-2.6.31-8-generic kernel.

After installing a new kernel check /boot/grub/grub.cfg and if the new kernel is missing run update-grub manually.

[David Favor (davidfavor)]

The following script can be used from a LiveCD to initially structure a GPT disk properly.

There still seems to be some problem where update-grub is never run when a new kernel is installed, even when correct GPT structure is in place.

Be sure to run update-grub after any kernel update for grub.cfg to contain the new kernel.

Best to try this script on a victim disk and be sure to change $dev, $fsdev and $mount_point to match your system.

Use this test script carefully to avoid data loss on a live disk.
_______

david@home1:~/bin$ cat gpt-setup-example
#!/bin/bash

# research all these options
# tune2fs -I 256 -O sparse_super,filetype,resize_inode,dir_index,ext_attr,has_journal,\
# extents,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize

if [ "$USER" != "root" ] ; then
   echo "Must run as USER=root instead of USER=$USER"
   exit 1
fi

dev=/dev/sdb
fsdev=/dev/sdb3
tune_opts='extents,uninit_bg,dir_index,journal_data_writeback'
mount_opts='noatime,data=writeback,barrier=0,nobh,errors=remount-ro'
mount_point=/data

mount $fsdev $mount_point -o $mount_opts 2>/dev/null
filecount=$(find $mount_point -type f 2>/dev/null | wc -l)

if [ "$filecount" -gt 0 ] ; then
   echo "Operation will destroy all data on $mount_point"
   echo "Remove all files from $mount_point before proceeding"
   exit 1
fi

umount $fsdev 2>/dev/null

set -x

umount $fsdev 2>/dev/null
mkdir -p $mount_point

parted -s $dev mklabel gpt

# must use <1M as parted (through version 1.8.8.1.159-1e0e) has a bug
# where 1M is either converted to 0M or rounded down to nearest block
# using kB units seems to circumvent this problem well
###parted -s $dev mkpart primary 0 1.1M
###parted -s $dev mkpart swap 1.1M 8G

parted -s $dev mkpart primary 0 1100kB
parted -s $dev mkpart swap 1100kB 8G
parted -s $dev mkpart primary 8G 28G

# tell Grub 2 MBR is big enough to use without blocklists as
# using blocklists seems to create a vast array of instabilities
parted -s $dev set 1 bios_grub on

parted -s $dev unit kB p

mkfs.ext4 $fsdev

tune2fs -L disk2 -m 1 -o $tune_opts $fsdev

e2fsck -pfD $fsdev

mount $fsdev $mount_point -o $mount_opts

mount -l | grep $fsdev

[Robert Millan (rmh-aybabtu)]

> This bug does not appear in 9.04 as I can install it just fine and appears to specifically relate to grub 2 + large drives.

Not exactly. Short explanation:

- Large drives (>2 TiB) need GPT instead of DOS-style partition labels.

- GRUB 2 supports GPT in two ways: a robust one, which requires the BIOS Boot partition, and the blocklist method which we don't recommend (see http://grub.enbug.org/BIOS_Boot_Partition for details). Even then, blocklist-based installs should work, although we can't really guarantee that they will continue to work after long periods of using the filesystem, or after doing certain things with it.

- GRUB Legacy doesn't support GPT at all. There's an unofficial patch, which is included by some distributions (including Ubuntu) which enables support for the blocklist method ONLY. This is, of course, far from optimal. But my understanding is that work is under way to get this properly sorted out in Karmic.

HTH

[Colin Watson (cjwatson)]

FWIW, the very last thing that grub-installer does is to set up kernel-img.conf to include update-grub, so if it crashes before that then that won't happen. (In general, if the installer crashes, then you can expect the installation to be incomplete in some ways, even if you attempt to work around the crashes.)

[David Favor (davidfavor)]

Thanks and yes. If there is a crash manually adding these lines to /etc/kernel-img.conf fixes further challenges.

postinst_hook = update-grub
postrm_hook = update-grub

[Evan (ev)]

David, given the fix in bug 417724, is this bug still present for you in the latest daily-live CDs (http://cdimage.ubuntu.com/daily-live)?

Thanks!

[Brian Murray (brian-murray)]

Looking at the attachments in this bug report, I noticed that "Installer" was flagged as a patch. A patch contains changes to an Ubuntu package that will resolve a bug, since this was not one I've unchecked the patch flag for it. In the future keep in mind the definition of a patch. You can learn more about what qualifies as a patch at https://wiki.ubuntu.com/Bugs/Patches. Thanks!

[Charlie Kravetz (cjkgeek)]

given the fix in bug 417724, is this bug still present for you in the latest daily-live CDs (http://cdimage.ubuntu.com/daily-live)?

Thanks!

[Charlie Kravetz (cjkgeek)]

Given that there has been no response, this bug is being left open for developers use. If the information should be supplied, that would really be a big help to insure this is resolved.

[Phillip Susi (psusi)]

Is anyone still seeing this on 12.04+?

[Paul White (paulw2u)]

This issue has sat with a status of 'Incomplete' for almost nine years without any response so it is being closed as 'Invalid'.

Please feel free to re-open this bug report if this is still an issue when using a currently supported release of Ubuntu and tell us in which release you still experience this issue.