[Karmic] Today's update of grub-pc left my computer non-bootable

[Roland Hughes]

I applied the 50 updates KPackageKit said needed to be applied today. After playing swat-the-mole clicking on APT failures I made the mistake of trying to do an orderly reboot. The GRUB-PC upgrade broke my machine. It will not boot.

symbol 'grub_zalloc' not found.

I don' t know how to operate from the GRUB_RECOVERY> prompt.

I had to file this bug report from a Windows PC.

[Felix Zielcke (fzielcke)]

Am Mittwoch, den 02.09.2009, 22:40 +0000 schrieb seasoned_geek:
> Public bug reported:
>
> I applied the 50 updates KPackageKit said needed to be applied today.
> After playing swat-the-mole clicking on APT failures I made the mistake
> of trying to do an orderly reboot. The GRUB-PC upgrade broke my
> machine. It will not boot.
>
> symbol 'grub_zalloc' not found.
>
> I don' t know how to operate from the GRUB_RECOVERY> prompt.
>
> I had to file this bug report from a Windows PC.
>

You must run grub-install on the device/disk you use for booting.
The embed core.img is now out of sync with the rest of /boot/grub.

Here's a guide how to do it from a Livecd like the Ubuntu one:
http://grub.enbug.org/Grub2LiveCdInstallGuide

[Roland Hughes (original-seasoned-geek)]

The problem is Karmic doesn't order the SATA drives properly. This pooches
FreeDOS, Grub, and everything else.

You are going to encounter a lot of people with more than one SATA drive,
Karmic had best get this bug sorted out.

On Thursday 03 September 2009 03:21:05 am Felix Zielcke wrote:
> *** This bug is a duplicate of bug 408699 ***
> https://bugs.launchpad.net/bugs/408699
>
> Am Mittwoch, den 02.09.2009, 22:40 +0000 schrieb seasoned_geek:
> > Public bug reported:
> >
> > I applied the 50 updates KPackageKit said needed to be applied today.
> > After playing swat-the-mole clicking on APT failures I made the mistake
> > of trying to do an orderly reboot. The GRUB-PC upgrade broke my
> > machine. It will not boot.
> >
> > symbol 'grub_zalloc' not found.
> >
> > I don' t know how to operate from the GRUB_RECOVERY> prompt.
> >
> > I had to file this bug report from a Windows PC.
>
> You must run grub-install on the device/disk you use for booting.
> The embed core.img is now out of sync with the rest of /boot/grub.
>
> Here's a guide how to do it from a Livecd like the Ubuntu one:
> http://grub.enbug.org/Grub2LiveCdInstallGuide
>
>
> ** This bug has been marked a duplicate of bug 408699
> karmic " 'grub_zalloc' not found " after update
>

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593 (cell)
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.logikalsolutions.com

[Colin Watson (cjwatson)]

Could you please provide the output of 'debconf-show grub-pc'?

[Roland Hughes (original-seasoned-geek)]

Keep in mind this is _after_ I used a Windows notebook to find out how to
manually fix this.

roland@logikaldesktop:~$ debconf-show grub-pc
debconf: DbDriver "passwords" warning: could not open
/var/cache/debconf/passwords.dat: Permission denied
  grub-pc/kopt_extracted: false
  grub2/kfreebsd_cmdline:
* grub-pc/install_devices: /dev/sda
  grub-pc/postrm_purge_boot_grub: false
  grub-pc/linux_cmdline: fillme
* grub2/linux_cmdline:
  grub2/kfreebsd_cmdline_default: quiet
* grub2/linux_cmdline_default: quiet splash
  grub-pc/chainload_from_menu.lst: true

I believe the root of the problem to be the fact Karmic does not see the SATA
drives in the correct order.

roland@logikaldesktop:~$ sudo fdisk -l
[sudo] password for roland:

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000553e3

   Device Boot Start End Blocks Id System
/dev/sda1 2 121601 976752000 f W95 Ext'd (LBA)
/dev/sda5 2 15666 125829081 83 Linux
/dev/sda6 15667 80936 524281243+ 83 Linux
/dev/sda7 80937 121601 326641581 83 Linux

Disk /dev/sdb: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0005713f

   Device Boot Start End Blocks Id System
/dev/sdb1 * 1 1020 8193118+ b W95 FAT32
/dev/sdb2 1021 30401 236002882+ f W95 Ext'd (LBA)
/dev/sdb5 1021 22931 176000076 b W95 FAT32
/dev/sdb6 22932 29010 48829536 83 Linux
/dev/sdb7 29011 30401 11173176 82 Linux swap / Solaris

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000cf7ab

   Device Boot Start End Blocks Id System
/dev/sdc1 1 121601 976760001 83 Linux

The BIOS sees the 250GB drive as the first and bootable drive. This is the way
it is cabled, there is no tweaking happening in the BIOS to change order.
FreeDOS was installed along with a large FAT-32 extended partition to act as a
transfer drive. Karmic book partition is also on that very same drive along
with the swap partition.

On Thursday 03 September 2009 11:09:06 am Colin Watson wrote:
> *** This bug is a duplicate of bug 408699 ***
> https://bugs.launchpad.net/bugs/408699
>
> Could you please provide the output of 'debconf-show grub-pc'?
>

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593 (cell)
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.logikalsolutions.com

[Colin Watson (cjwatson)]

So, it's definitely true that /dev/sda is not a reliable way to identify
any given disk. We moved to UUIDs for mounting filesystems from
partitions precisely because such names are unreliable. The kernel
believes itself entitled to enumerate SATA (and many other) devices in
any order it pleases - indeed, it has no choice on some SCSI systems, or
given that the enumeration namespace is shared with USB, etc. - and it
will frequently not use the order you might expect. Applications are
supposed to cope in other ways. This has been an ongoing headache in
various places for years - it's not unique to GRUB 2 in Karmic by any
stretch of the imagination, though you might have been lucky enough to
have escaped its deathly touch until now.

That said, UUIDs are a property of filesystems, not of disks. The only
reliable and suitable way I can think of to identify disks is to use the
names under /dev/disk/by-id/ rather than the traditional Linux device
names; those essentially identify the general type (e.g. SATA), the
manufacturer's name, and a unique serial number for the disk, packed
into a device name format. In short they uniquely identify a given
physical disk, or a numbered partition on a given physical disk, but not
any kind of logical construct on that disk (so they aren't suitable for
filesystem mounting where you care much more about the contents). See
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=509378#24 for a related
discussion.

Felix, if you're reading, what do you think of the following approach?

 * If the directory /dev/disk/by-id exists (it should more or less
   always exist if you're using udev), then always store device names
   under /dev/disk/by-id/ in grub-pc/install_devices. Map them back to
   the traditional names as found in device.map before using them.

 * It's probably not suitable to actually display /dev/disk/by-id/ names
   to users, e.g. in the list of choices for that template. Instead,
   display either the traditional Linux device name, or (preferably)
   some kind of "friendly" name, including such things as the size of
   the disk. See d-i's partitioner for a partial model here.

 * grub-installer should do this translation too.

 * On upgrade to the first version implementing this, if /dev/disk/by-id
   exists and grub-pc/install_devices does not already contain suitable
   names (the latter check may be unnecessary, as more or less nobody's
   will), *ask again*, possibly with different text. The reason for this
   is that their stored list of install devices might well be incorrect,
   and we want to make sure that we get it accurate as soon as we can
   store it accurately, and then store it accurately henceforth.

 * If ever it is discovered that the stored list of install devices
   includes a disk that no longer exists, ask the question again. This
   is probably because a disk was swapped out for a different one, and
   we almost certainly need new information on the state of the system.
   In this case especially, we may need to regenerate device.map; in
   general I think grub-setup falls over if it's asked to install to a
   disk not listed in device.map, although for our purposes it doesn't
   mat...

[Roland Hughes (original-seasoned-geek)]

I can tell you this much.

When FreeDOS is installed on the first partition of the first BIOS recognized
SATA drive, it needs to still be bootable. Prior to Karmic, this was always
possible. After Karmic it is no longer possible. Any fix you propose must
work in that manner.

Personally, I think the low level should respect the BIOS ordering of the
drives. You only need to do this once at bootup, even if you don't use the
information for anything other than grub.

What you also haven't taken into account are stand alone backup utilities.
Image for Linux now has this same disaster because they upgraded the kernel
used for their bootable CDs. The old Kernel honored all BIOS orders. Now,
when you back up the first drive Image sees, it isn't the first drive you BIOS
sees.

A general user is smart enough to learn how to run backup software. They
aren't smart enough to realize their disk drives are changing order. I have 3
SATA drives, two of which are identical. If it wasn't for the fact I created
a different partition map on each one, I would have absolutely NO method of
telling them apart.

While you are fixing this, you need to enhance the "support" provided so that
partition labels are viewable everywhere via every utility which shows a
partition. Been burned by that many times.

On Thursday 03 September 2009 07:34:24 pm Colin Watson wrote:
> *** This bug is a duplicate of bug 408699 ***
> https://bugs.launchpad.net/bugs/408699
>
> So, it's definitely true that /dev/sda is not a reliable way to identify
> any given disk. We moved to UUIDs for mounting filesystems from
> partitions precisely because such names are unreliable. The kernel
> believes itself entitled to enumerate SATA (and many other) devices in
> any order it pleases - indeed, it has no choice on some SCSI systems, or
> given that the enumeration namespace is shared with USB, etc. - and it
> will frequently not use the order you might expect. Applications are
> supposed to cope in other ways. This has been an ongoing headache in
> various places for years - it's not unique to GRUB 2 in Karmic by any
> stretch of the imagination, though you might have been lucky enough to
> have escaped its deathly touch until now.
>
> That said, UUIDs are a property of filesystems, not of disks. The only
> reliable and suitable way I can think of to identify disks is to use the
> names under /dev/disk/by-id/ rather than the traditional Linux device
> names; those essentially identify the general type (e.g. SATA), the
> manufacturer's name, and a unique serial number for the disk, packed
> into a device name format. In short they uniquely identify a given
> physical disk, or a numbered partition on a given physical disk, but not
> any kind of logical construct on that disk (so they aren't suitable for
> filesystem mounting where you care much more about the contents). See
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=509378#24 for a related
> discussion.
>
> Felix, if you're reading, what do you think of the following approach?
>
> * If the directory /dev/disk/by-id exists (it should more or less
> always exist if you're using udev), then always store dev...

[Felix Zielcke (fzielcke)]

Am Freitag, den 04.09.2009, 00:34 +0000 schrieb Colin Watson:

>
> Felix, if you're reading, what do you think of the following approach?
>

Sounds good. Robert already said once that the /dev/disk/by-id links
look good.
So please go ahead.

[Colin Watson (cjwatson)]

In general, it is not possible to even discover the BIOS ordering from
Linux. This perhaps non-obvious constraint (beyond the control of GRUB)
makes all kinds of things rather difficult. I have banged my head
against this fruitlessly in the past, and have no intention of wasting
very much time on it again; it was, mostly, a blind alley. Dell have a
set of BIOS extensions called EDD which let you do a little better, and
it might be worth supporting those in GRUB's device.map code, but
they're only supported on a fraction of today's PC-class computers.

Furthermore, BIOS ordering is unstable (in particular, consider
removable drives), so we can't rely on it, convenient as it might be in
some scenarios. This is why we normally use UUIDs when booting operating
systems from GRUB, and in general try to avoid relying on BIOS ordering
at all. However, FreeDOS probably doesn't support this kind of scheme.
I'll endeavour to test FreeDOS in a multi-drive test system. I don't
think that it is directly related to the things I'm trying to discuss
with Felix here, although fixing those is probably a prerequisite for
making this work reliably.

Your comments about Image for Linux (not a product I'm familiar with)
are presumably simply due to them using a new kernel which enumerates
devices in a different order. I haven't taken this into account because,
while I'm sure it is a bug, it is not my problem; I'm afraid that this
is beyond my control and nothing to do with GRUB. No doubt the
application needs to improve how it displays disks.

I'd appreciate a *separate* bug report (on partman-base) about
displaying volume labels in the installer's partitioner. Let's try to
keep this bug report somewhat under control in terms of scope so that we
can fix it.

[Felix Zielcke (fzielcke)]

Am Freitag, den 04.09.2009, 08:27 +0000 schrieb Colin Watson:
> Dell have a
> set of BIOS extensions called EDD which let you do a little better,
> and
> it might be worth supporting those in GRUB's device.map code, but
> they're only supported on a fraction of today's PC-class computers.

I don't know how well EDD is today supported, but if I looked correctly
the kernel which handles this works only when it's loaded with the
16bit entry point and not the 32bit one.
So I don't think it's worth at all now that we use by default the 32bit
entry point.

--
Felix Zielcke
Proud Debian Maintainer

[Colin Watson (cjwatson)]

That wouldn't surprise me - EDD needs real-mode access. It's probably
best to continue our efforts to avoid the problem altogether.

[Roland Hughes (original-seasoned-geek)]

I attached a comment to 22756 where they had it flagged for wishlist, but given
the viciousness of the SATA drive order bug, being able to see
partition/volume labels has become critical.

On Friday 04 September 2009 03:27:31 am Colin Watson wrote:
> *** This bug is a duplicate of bug 408699 ***
> https://bugs.launchpad.net/bugs/408699
>
> In general, it is not possible to even discover the BIOS ordering from
> Linux. This perhaps non-obvious constraint (beyond the control of GRUB)
> makes all kinds of things rather difficult. I have banged my head
> against this fruitlessly in the past, and have no intention of wasting
> very much time on it again; it was, mostly, a blind alley. Dell have a
> set of BIOS extensions called EDD which let you do a little better, and
> it might be worth supporting those in GRUB's device.map code, but
> they're only supported on a fraction of today's PC-class computers.
>
> Furthermore, BIOS ordering is unstable (in particular, consider
> removable drives), so we can't rely on it, convenient as it might be in
> some scenarios. This is why we normally use UUIDs when booting operating
> systems from GRUB, and in general try to avoid relying on BIOS ordering
> at all. However, FreeDOS probably doesn't support this kind of scheme.
> I'll endeavour to test FreeDOS in a multi-drive test system. I don't
> think that it is directly related to the things I'm trying to discuss
> with Felix here, although fixing those is probably a prerequisite for
> making this work reliably.
>
> Your comments about Image for Linux (not a product I'm familiar with)
> are presumably simply due to them using a new kernel which enumerates
> devices in a different order. I haven't taken this into account because,
> while I'm sure it is a bug, it is not my problem; I'm afraid that this
> is beyond my control and nothing to do with GRUB. No doubt the
> application needs to improve how it displays disks.
>
> I'd appreciate a *separate* bug report (on partman-base) about
> displaying volume labels in the installer's partitioner. Let's try to
> keep this bug report somewhat under control in terms of scope so that we
> can fix it.
>

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593 (cell)
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.logikalsolutions.com