update to grub-pc writes MBR without checks, prompt or backup

[Adrian Wilkins]

Binary package hint: grub

Booted laptop from external USB HDD containing Karmic.

Installed updates. One of the updates is to grub.

Internal drive contains a full-disk encrypted Windows installation.

Update to grub writes the grub MBR to the internal disk, instead of the disk that grub booted the original MBR from.

On a standard Windows install I would have considered this merely annoying, because I could have replaced the MBR easily enough. On this install I have to get one of the support techs to "bless" the disk with a special bootloader and the "Code of the Day".

This is a more general case of #112239 "GRUB writes to wrong MBR and destroys RAID setup"

Suggestions ;

 - Where multiple disks present, always prompt the user for the disk to put the MBR on. This would help prevent these problems
   - Make the prompt nice and clear, disk descriptors, sizes, etc
 - When UPDATING grub, only write the MBR to a disk that has a recognized grub MBR on it already.
   - It's not an "install" (and be damned), it's an update
 - Whenever you write any MBR, write the previous MBR data to a permanent log file. This way it can be restored afterwards.

[Adrian Wilkins (adrian-wilkins)]

Found the bit of the apt log.

Oh, I'd rate this as "Critical" in terms of importance (although I'm biased). Things that are "routine" system updates should never do potentially system-wrecking things. I felt slightly weird whenever I saw that "beta" version moniker on the boot screen.. and now I know why.

===================

Setting up grub-pc (1.97~beta4-1ubuntu4.1) ...
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(hd0) /dev/sda
(hd1) /dev/sdb
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-16-generic
Found initrd image: /boot/initrd.img-2.6.31-16-generic
Found linux image: /boot/vmlinuz-2.6.31-15-generic
Found initrd image: /boot/initrd.img-2.6.31-15-generic
Found linux image: /boot/vmlinuz-2.6.31-14-generic
Found initrd image: /boot/initrd.img-2.6.31-14-generic
Found memtest86+ image: /boot/memtest86+.bin
done

[Adrian Wilkins (adrian-wilkins)]

Now I discover that this has trashed the MBR on my desktop workstation as well (so I'm setting the status to "Confirmed") ; not only have there been updates to the package with the bootloader in, there have been MULTIPLE updates (one of which trashed the laptop, one of which has trashed the desktop). This has been deeply inconvenient.

A dist-update from Jaunty to Karmic doesn't upgrade grub to grub2 (AFAIK), so why should a fresh install be different and upgrade the version of grub?

I've added the following file to my config... hopefully this will suppress the package updates and thus this behaviour, but I'll be keeping a sharp eye on the updates list in future.

# /etc/apt/apt.conf.d/02holdgrub

HoldPkgs { "grub-pc"; };

[Felix Zielcke (fzielcke)]

Am Montag, den 14.12.2009, 13:45 +0000 schrieb Adrian Wilkins:
>
> A dist-update from Jaunty to Karmic doesn't upgrade grub to grub2
> (AFAIK), so why should a fresh install be different and upgrade the
> version of grub?

Well Ubuntu choose the way to use GRUB 2 for new installs, but to not
change the GRUB version for updates from jaunty (or older versions).
This is more safe because GRUB Legacy supports some setups like dmraid
which GRUB 2 didn't support at all, when karmic was released. And in
these cases grub-installer still installs GRUB Legacy.

The package grub-pc itself asks via Debconf for the device it should run
grub-install onto.
But this value gets preseeded by grub-installer on new installs, with
the device where it installs GRUB to.

If you want to change or remove it run: `sudo dpkg-reconfigure grub-pc'
But if the package gets upgraded and update-grub generates a grub.cfg
which isn't 100% compatible with the GRUB version in /boot/grub and
embeding area then maybe you can't boot. That's why we implemented this.
--
Felix Zielcke
Proud Debian Maintainer and GNU GRUB developer

[Adrian Wilkins (adrian-wilkins)]

And I right in saying that the device that grub boots is identified by UUID and the device that it installs grub to is identified by a path in /dev ?

[Felix Zielcke (fzielcke)]

Am Montag, den 14.12.2009, 22:16 +0000 schrieb Adrian Wilkins:
> And I right in saying that the device that grub boots is identified by
> UUID and the device that it installs grub to is identified by a path
> in
> /dev ?
>

The GRUB device which gets used during boot is only searched with UUID
if you don't install to the disk which contains your /boot/grub.
Else the partition number gets hardcoded and drive number is read from
BIOS.
The script grub-install supports actually both, a GRUB device and an OS
one, i.e. /dev/XXX
But it's not recommended to use a GRUB device because that totally
depends on /boot/grub/device.map and that could be completely wrong.
--
Felix Zielcke
Proud Debian Maintainer and GNU GRUB developer

[MGHG (mghg)]

Almost the same sequence of events as in the bug description happended to me on the evening of 15th December 2009.

In my case it was a USB flash memory containing Ubuntu 9.10 Karmic and not a HDD, but I assume that has no significance.

The bug has had a critical impact for me since it was necessary to
1. format the harddrive and re-install Windows
2. install standard and company-specific applications
3. encrypt the harddrive
4. finalize with personal finetuning

In addition, I can not risk to boot my laptop from an external USB flash memory containing Ubuntu if the updates are so unreliable and may cause several days of extra work.

[Christian Bejram (christian-bejram)]

Hello,

This very same issue affected me January 1st 2011.

My setup is:
 - a laptop with HDA encrypted using SafeBoot (require a SafeBoot MBR to work)
 - Ubuntu 10.04 LTS 64-bit version on an external USB disk (this becomes HDB/SDB)
   Selecting using BIOS boot menu whether to boot using internal disk or external disk. (this works very well)

On January 1st I received an system update of grub-pc (and other packages), accepted and installed all of them.
After the system update it was no longer possible to boot on the internal HD.
Discovered that grub-pc update had trashed the MBR of the internal HD.

Had to do a SafeBoot MBR recovery (which is a bit cumbersome) to get the system back to normal again.

To me grub/grub2/grub-pc is viral, it should not overwrite and destroy the MBR without asking.

// Christian

[Ronan Mooney (roomey)]

Same issue affected my, already filled a duplicate bug as I did not see this one. Details are:

Hard drive has the following partitions-

SDA1 FAT 16 Dell recover partition
SDA 2 NTFS windows recovery partition
SDA3 window (truecrypt encrypted system drive)
SDA4 Extended
SDA5 EXT4 Ubuntu 10.04
SDA6 SWAP.

To set up a dual boot encrypted system I did the following
Encrypted the Windows system partition using trucrypt- this overwrote the MBR with the truecrypt bootloader.
I then used a rescue disk to install grub 2 on SDA1
Then I marked SDA1 as bootable.

The following was the behaviour before the GRUB2 upgrade today.

At boot, truecrypt boot screen appeared-
Option one- enter windows passphrase- windows then booted
Option two- press escape, Then the Grub boot menu appeared allowing me to to ubuntu.

Post update behaviour.
Grub2 update overwrote MBR without confirmation, and as far as I can see, without taking a copy of the MBR)
At Boot, Grub only lists ubuntu boot options (as windows partition is encrypted.

Attempted solution:
Reinstall Trucrypt bootloader from rescue disk. This was not successful, as Trucrypt boot loader only boots whichever partition has a "boot Flag". This means I can only access windows.

Expected behaviour:
Grub update will only overwrite MBR if grub is installed there already.
Or, Grub asks for confirmation before overwriting MBR
Or, Copy of MBR is taken before Grub overwrites it (this would have saved me a lot of time)

As of now, to fix, I have had to totally decrypt windows partition and re-encrypt and do the boot loaders swap. I will also have to blocklist grub2.

Comment from Colin Watson is:

There's no reliable way to tell whether GRUB is installed in an MBR or not; asking for confirmation every time would be an awful lot of confirmation prompts for most people. We have talked about keeping a backup and may do this in the future, though.

[Ronan Mooney (roomey)]

Colin,
There's no reliable way to tell whether GRUB is installed in an MBR or not-
This seems unlikely. I am sure a "best Guess" could be made, at which point a confirmation question could be asked.

About asking for confirmation each time- Yes I can see this may cause problems

Abut creating a copy- I think this is an urgent, graceful solution (particularly as MBR is small), and should be put in place.

I would support this bug being marked critical, as it is an upgrade which breaks an install of multiple systems which can take a long time to restore.

[Colin Watson (cjwatson)]

On Fri, Feb 04, 2011 at 12:36:31PM -0000, Ronan Mooney wrote:
> There's no reliable way to tell whether GRUB is installed in an MBR or
> not-
> This seems unlikely. I am sure a "best Guess" could be made, at which
> point a confirmation question could be asked.

Unlikely or not, it's my assessment that it's true. Matching binary
code would be very fragile, would have to detect all old versions of
GRUB, would be liable to break any time GRUB is changed, and there's no
way I can think of to tell which of the many other MBR implementations
would share some of that code. Matching text strings suffers from the
problem that the GRUB-specific text strings in the MBR are relatively
close to the end of the MBR, and it's entirely possible that an MBR
implementation that didn't use the whole available space wouldn't
overwrite them. Thus I believe any such check would be unreliable in
both directions.

[Ronan Mooney (roomey)]

Thanks Colin, That does make perfect sense.

So it seems like the only, and easiest solution is for the grub update script to create a copy of the MBR before it is overwritten, and for instructions regarding moving the grub install to remind people to run sudo dpkg-reconfigure grub-pc

[Marco Cimmino (cimmo)]

kubuntu 11.04 beta2
1. installed windows
2. installed kubuntu 11.04 and encrypted
3. encrypted windows using pgp

everything was fine until grub2 update kicked in and overwrote mbr, so now I cannot see windows encrypted partition anymore in grub2, this is easy to fix, but the problem that pgp also doesn't ask anymore for password at startup.

This is bad.

[Adrian Wilkins (adrian-wilkins)]

Would it be easier to detect other bootloaders?

I would guess that most common suspects do not change frequently ; AFAIK Windows never updates the MBR automatically after installation.

There may be some characteristics of the various encrypted partition bootloaders that can also be readily recognised.

[WSC (ohir)]

I can confirm this bug (looks more like plain stupidity than a bug) and it affected me badly.

Never ever update of anything should touch MBR or 'autoidiotically' change vital system areas
unless explicitly told to do so by admin.

> Would it be easier to detect other bootloaders?

No. No one can know of all or even most of bootloaders in use, the less how to detect them reliably.
Ie there can be govt mandated security assesment under its own bootmanager that a few people
will ever see.

Fix:

There is /boot/grub or /etc/default/grub to use for. flag whether messing with MBR is allowed.
If user/admin agrees to during install, put either grub_can_mess_with_mbr.yes file into /boot/grub directory
or fill in GRUB_CAN_MESS_WITH_MBR=yes in suitable script in /etc/grub.d and transfer it to grub.cfg
for inspection. also GRUB_INSTALL_TO_DEV= or GRUB_INSTALL_TO_UUID= need to be introduced
and RESPECTED.

P.S. It is CRITICAL bug. In fact due to this stupidity of grub2 update I am told now to purge ubuntu from
ALL places in my organization, after a few year battle for being allowed to install ubuntu on less important
production machines.

System: 10.04 LTS / dual boot with nationa

[Marco Cimmino (cimmo)]

Is there a way to get this prioritized? Come on guys... wiping out MBR without asking...

[Brad (braddeicide)]

During the installer grub can be installed onto a partition instead of MBR, however during updates it ignores this installation option and overwrites the MBR? There should be a GRUB_DEVICE setting saved somewhere and checked by the grub update script otherwise having the option in the installer is wrong as it creates a grub installation which is incompatible with ubuntu going forward.

Due to grubs inability to boot truecrypt (https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/484102) this bug effects all users of encrypted ubuntu+windows truecrypt systems.

[Phillip Susi (psusi)]

Collin, couldn't debconf store the md5sum of the MBR ( after zeroing out the partition table ), and on upgrade, verify that it still matches, or prompt for confirmation?

[Adrian Wilkins (adrian-wilkins)]

I upgraded grub on a VM which I had snapshotted - apparently, this changes the disk ID.

This was on Precise Server. I was prompted for which drive to reinstall grub to, which is pretty much the desired result - grub is not installing itself to a disk it hasn't seen before.

Can I infer this has been fixed, or does it not work with the desktop distribution? I may have to make a VM and try the same trick...

[Jeremy Visser (jeremy-visser)]

Not intending to troll, but...

On 18/11/2012 15:30, Phillip Susi wrote:
> ** Changed in: grub2 (Ubuntu)
> Status: Confirmed => Fix Released

[citation needed]

[Phillip Susi (psusi)]

As the previous comment indicated, grub checks to make sure the drives are the same that were selected before, and if not, prompts again.

[Jeremy Visser (jeremy-visser)]

On 19/11/2012 02:07, Phillip Susi wrote:
> As the previous comment indicated, grub checks to make sure the drives
> are the same that were selected before, and if not, prompts again.

I was thinking more along the lines of linking to Debian changelog
entries and specifying the actual package names/versions that the fix
was released in, rather than anecdotes.

[Jeremy Visser (jeremy-visser)]

I'm running a laptop dual-booting Windows 7 with TrueCrypt (thus TrueCrypt MBR) and Ubuntu 10.04.

I just upgraded it to Ubuntu 12.04 using the GUI dist-upgrade tool, and it overwrote the unique TrueCrypt MBR, rendering the Windows 7 installation useless. I don't have the recovery disc, so all my data on Windows was lost

This bug is most certainly not fixed, at least in 12.04.

[Jeremy Visser (jeremy-visser)]

On 27/11/12 12:26, Jeremy Visser wrote:
> I just upgraded it to Ubuntu 12.04 using the GUI dist-upgrade tool, and
> it overwrote the unique TrueCrypt MBR, rendering the Windows 7
> installation useless.

And just so Google doesn't forever remember me as the idiot who upgraded
without backups, this *was* a throwaway laptop so losing all the Windows
data is an inconvenient setback, but nothing more.