Grub2 affected when os-prober incorrectly identifies Windows and Recovery partitions

[DaveR]

Binary package hint: grub

The grub os-prober mis-identifies the Compaq recovery partition as Windows XP and the Windows XP partition as "ubuntu linux". Choosing Windows XP from the grub boot menu runs the Compaq recovery program - a BAD idea. Choosing ubuntu linux on the other hand launches Windows XP correctly (or at least it did before today's update!). Substantiating information below:

Grub was updated 6Nov 10:30AM but that doesn't change the behavior from before the update.

<<< Output from fdisk -l with added comment identifying partition usage >>>

   Device Boot Start End Blocks Id System
/dev/sda1 1 617 4664488+ b W95 FAT32 Compaq recovery partition
/dev/sda2 * 618 5372 35947800 7 HPFS/NTFS Windows XP
/dev/sda3 5373 7269 14341320 83 Linux Ubuntu /
/dev/sda4 7270 20671 101319120 f W95 Ext'd (LBA)
/dev/sda5 7270 7404 1020568+ 82 Linux swap / Solaris /swap
/dev/sda6 7405 20671 100298488+ 83 Linux /home

<<< Partial content of /boot/grub/grub.cfg showing relevant sections >>>

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows NT/2000/XP (on /dev/sda1)" {
 insmod fat
 set root=(hd0,1)
 search --no-floppy --fs-uuid --set 4452-51a8
 drivemap -s (hd0) ${root}
 chainloader +1
}
menuentry "ubuntu Linux (on /dev/sda2)" {
 insmod ntfs
 set root=(hd0,2)
 search --no-floppy --fs-uuid --set 3c78b47778b4318e
 drivemap -s (hd0) ${root}
 chainloader +1
}
### END /etc/grub.d/30_os-prober ###

<<< Partial list of grub.cfg generated by grub before today's update >>>

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows NT/2000/XP (on /dev/sda1)" {
 insmod fat
 set root=(hd0,1)
 search --no-floppy --fs-uuid --set 4452-51a8
 drivemap -s (hd0) ${root}
 chainloader +1
}
menuentry "ubuntu Linux (on /dev/sda2)" {
 insmod ntfs
 set root=(hd0,2)
 search --no-floppy --fs-uuid --set 3c78b47778b4318e
 drivemap -s (hd0) ${root}
 chainloader +1
}
### END /etc/grub.d/30_os-prober ###

ProblemType: Bug
Architecture: i386
Date: Fri Nov 6 10:45:25 2009
DistroRelease: Ubuntu 9.10
InstallationMedia: Ubuntu 9.10 "Karmic Koala" - Release i386 (20091028.5)
NonfreeKernelModules: nvidia
Package: grub (not installed)
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.48-generic
SourcePackage: grub
Uname: Linux 2.6.31-14-generic i686
XsessionErrors:
 (gnome-settings-daemon:1705): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (gnome-settings-daemon:1705): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (polkit-gnome-authentication-agent-1:1752): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (nautilus:1737): Eel-CRITICAL **: eel_preferences_get_boolean: assertion `preferences_is_initialized ()' failed
 (gnome-appearance-properties:1832): Gdk-CRITICAL **: gdk_display_sync: assertion `GDK_IS_DISPLAY (display)' failed

[DaveR (djrpublic)]

I forgot to mention that previous versions of grub displayed identical behavior. If I recall correctly I first partitioned this drive to install Ubuntu 5.??. I then learned to edit grub's menu manually to address the problem.

[Niels Kristian Bech Jensen (nkbjensen)]

I see the same behaviour on an Acer Aspire 7736ZG.

Output from 'fdisk -l /dev/sda':

/dev/sda1 1 1530 12288000 27 Unknown
/dev/sda2 * 1530 13280 94382576 7 HPFS/NTFS
/dev/sda3 13281 14065 6305512+ 83 Linux
/dev/sda4 14066 60801 375406920 5 Extended
/dev/sda5 14066 15110 8393931 82 Linux swap / Solaris
/dev/sda6 15111 60801 367012926 83 Linux

Part of /boot/grub/grub.cfg:

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows Vista (loader) (on /dev/sda1)" {
        insmod ntfs
        set root=(hd0,1)
        search --no-floppy --fs-uuid --set daa41c49a41c2b11
        chainloader +1
}
menuentry "Windows 7 (loader) (on /dev/sda2)" {
        insmod ntfs
        set root=(hd0,2)
        search --no-floppy --fs-uuid --set 640c1b9c0c1b6876
        chainloader +1
}
### END /etc/grub.d/30_os-prober ###

[TJ (tj)]

I'm seeing a similar problem with Lucid.

$ sudo fdisk -l /dev/sda
...
   Device Boot Start End Blocks Id System
/dev/sda1 1 1217 9775521 27 Unknown
/dev/sda2 * 1218 4796 28748317+ 7 HPFS/NTFS
/dev/sda3 4797 4829 265072+ 83 Linux
/dev/sda4 4830 48641 351919890 8e Linux LVM

In this case:
 sda1 is the original Sony Windows Vista Recovery/Re-installation partition.
 sda2 is Windows Vista
 sda3 is the Linux /boot/ partition
 sda4 is the LVM PV containing numerous logical volumes.

/boot/grub/grub.cfg contains:

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows Vista (loader) (on /dev/sda1)" {
 insmod ntfs
 set root='(hd0,1)'
 search --no-floppy --fs-uuid --set cce61747e61730e6
 chainloader +1
}
menuentry "Windows Recovery Environment (loader) (on /dev/sda2)" {
 insmod ntfs
 set root='(hd0,2)'
 search --no-floppy --fs-uuid --set e6e08581e0855927
 drivemap -s (hd0) ${root}
 chainloader +1
}
### END /etc/grub.d/30_os-prober ###

When update-grub is running during the package upgrade the console shows:

Found Windows Vista (loader) on /dev/sda1
Found Windows Recovery Environment (loader) on /dev/sda2
done

[TJ (tj)]

The bug is in /usr/lib/os-probes/mounted/20microsoft

When probing the Boot Configuration Data binary file it makes several invalid assumptions:

1. All recovery file-systems are named (in UTF-16) "Windows Recovery Environment"
2. The last entry in the /Boot/BCD file is related to the partition being probed
3. Any BCD it can't 'understand' must be "Windows Vista"

In my case /dev/sda1 (the recovery partition) has a /boot/bcd containing "Vaio Recovery Environment" and "Windows Memory Diagnostic", neither of which will match:

 elif grep -qs "W.i.n.d.o.w.s. .R.e.c.o.v.e.r.y. .E.n.v.i.r.o.n.m.e.n.t" "$2/$boot/$bcd"; then
  long="Windows Recovery Environment (loader)"

resulting in the default being used:

 else
  long="Windows Vista (loader)"

For /dev/sda2 (the Windows Vista partition) has a /Boot/BCD containing "Windows Vista" followed by "Windows Recovery Environment". In this case because there is no specific test for "Windows Vista" a match is made with:

 elif grep -qs "W.i.n.d.o.w.s. .R.e.c.o.v.e.r.y. .E.n.v.i.r.o.n.m.e.n.t" "$2/$boot/$bcd"; then
  long="Windows Recovery Environment (loader)"

and the default 'else' clause that would set "Windows Vista" is never used.

[TJ (tj)]

I've written a POSIX shell script that can reliably identify the name of the current default* BCD boot menu option in any bootable Windows partition that uses a Boot Configuration Data store.

It is based on the fact that a BCD store is a standard Windows registry hive, and there is now a LGPL v2.1 library available to reliably read registry hives.

If affected users can test it and report back using the instructions below, it might be a candidate for reliable detection in future Ubuntu releases.

Here's an example of (verbose) output on the system I described earlier:

$ DEBUG=1 ./probe_bcd /dev/sda1 /tmp/sda1/boot/bcd
Debug: Partition: sda1
Debug: BCD store: /tmp/sda1/boot/bcd
Debug: Using temporary XML file: /tmp/sda1.bcd.xml
Debug: sda1 default boot entry is GUID {7619dcc9-fafe-11d9-b411-000476eba25f}
Debug: "VAIO Recovery Environment" is the default boot entry (GUID: {7619dcc9-fafe-11d9-b411-000476eba25f})
VAIO Recovery Environment

$ DEBUG=1 ./probe_bcd /dev/sda2 /tmp/sda2/Boot/BCD
Debug: Partition: sda2
Debug: BCD store: /tmp/sda2/Boot/BCD
Debug: Using temporary XML file: /tmp/sda2.bcd.xml
Debug: sda2 default boot entry is GUID {21443b0f-f13b-11dd-9361-9de0f6f6dffb}
Debug: "Windows Vista (TM) Home Premium (recovered) " is the default boot entry (GUID: {21443b0f-f13b-11dd-9361-9de0f6f6dffb})
Windows Vista (TM) Home Premium (recovered)

The "Debug" lines are purely for testing and are written to STDERR. Without DEBUG the only output is the default boot entry name:

$ ./probe_bcd /dev/sda1 /tmp/sda1/boot/bcd
VAIO Recovery Environment

$ ./probe_bcd /dev/sda2 /tmp/sda2/Boot/BCD
Windows Vista (TM) Home Premium (recovered)

The script is designed to be called to assign a value to a variable, e.g:

 long=probe_bcd $partition $bcd

The script depends upon sed and gawk. It achieves its results by converting the BCD store, which is a Windows registry hive, to XML using the (currently unpackaged) library "hivex", part of the "libguest" group of libraries and tools.

http://libguestfs.org/hivexml.1.html
http://libguestfs.org/

Debian currently has a Request For Packaging bug filed for hivex to which I've responded with an Intent To Package (ITP) and have made an initial upload to Debian Mentors:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573070

hivexml (the BCD to XML converter) depends on libc6 and libxml2 at a minimum, with other dependencies if OCCAM and Perl support are required for other tools in the library.

I built hivex locally to check it would work with my "bcd_probe" script but now I've proved it works I have created a Debian/Ubuntu package which is available from my PPA.

If you'd like to test this against a system where the probing is guessing incorrectly, download the attached script files probe_bcd and get_node_name.gawk, install hivex from my PPA, and do something like this:

#----- BEGIN -----
# determine which architecture the system uses
ARCH=$(apt-get -v | head -n1 | cut -d" " -f 4)
# install the hivex package
wget https://launchpad.net/~intuitivenipple/+archive/ppa/+files/libhivex_1.2.1-1_${ARCH}.deb
wget https://edge.launchpad.net/~intuitivenipple/+archive/ppa/+files/hivex_1.2.1-1_${ARCH}.deb
dpkg -i libhivex_1.2.1-...

[Daniel Bair (danielbair)]

I have two Toshiba Satellite laptops and grub misidentifies the Recovery versus Vista partitions.
I can confirm that the script from comment #7 above does work correctly for me.

BCD store: /tmp/sda3/BOOT/BCD
Windows Setup
BCD store: /tmp/sda2/Boot/BCD
Microsoft Windows Vista

[Lars Adermalm (lars-adermalm)]

I have a Sony Vaio VGN-FZ19VN where dev/sda1 is my recovery partition and dev/sda2 is the Windows Vista partition. With karmic both were identified as Windows Vista (loader). The following is the output from grub2 in lucid beta 2:

lars@lars-laptop:~$ sudo update-grub
[sudo] password for lars:
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32-21-generic
Found initrd image: /boot/initrd.img-2.6.32-21-generic
Found Windows Vista (loader) on /dev/sda1
Found Windows Recovery Environment (loader) on /dev/sda2
done

[amay82 (andimayer82-deactivatedaccount)]

In my case (Amilo Pi3514), the Amilo Vista recovery partiotion is detected as "Windows Vista (loader)" while the "real" Windows is detected as "Windows Recovery Environment (loader)":

redforce@valhalla:/boot/grub$ sudo update-grub2
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32-21-generic
Found initrd image: /boot/initrd.img-2.6.32-21-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Windows Vista (loader) on /dev/sda1
Found Windows Recovery Environment (loader) on /dev/sda2
done

(should be swapped)

Can I provide more info? Please tell me what.

[Sergio Callegari (callegar)]

Same problem here. On a Dell laptop, Windows Vista is detected as a Windows recovery environment. The dell recovery partition is not seen at all.

Please _rise_ priority. This thing is driving _crazy_ people who try linux for the first time and get the impression that lucid has wiped their windows environment.

[R.Eric VanNewkirk (evannewkirk)]

Seeing the exact same problem as sergio on a Dell Studio 17 with two internal hard drives after the upgrade from 9.10 to Lucid. Vista, which lives on sda1, is misidentified as Windows Recovery Environment and the recovery partition isn't seen at all. Had me a little freaked out first time I saw it but Vista seemed to boot normally. Running update-grub doesn't fix the problem and grub.cfg says it's not to be edited so I'm not going to touch it.

Any help?

[TJ (tj)]

This may be fixed permanently for 10.10 Maverick. I've now got hivex ready for Debian which should make its way automatically into the Ubuntu archives. *If* that happens os-prober can use my probe_bcd script to correctly identify the Windows partitions.

For 10.04 Lucid I'm not sure how it could be solved, short of backporting hivex into the main archive. Hopefully one of the core developers and installer experts will take a look at this now the pressure from Lucid is off.

[Chris Koller (cjkhome)]

I have the same problem on my acer laptop. The vista partition has been incorrectly identified as windows recovery environment although windows does run ok. This problem has only occurred after upgrading from KK to LL.

[Jeff B (blaisdje)]

Ditto.

Dell Inspiron 1720

Dual Boot: Windows Vista | Ubuntu 10.04

Prior to updating to Lucid Lynx I was running Ubuntu Karmic Koala 9.10. No problems. Dual boot worked great.

After upgrade to 10.04, sda1 is being listed as the Windows Recovery partition. Should be Windows Vista. Windows Recovery would be sda2.

[Michael Kreuzer (penulci)]

Dell Vostro 410

Dual Boot: Windows Vista Business (32bit) | Ubuntu 10.04 (64bit)

Vista on /dev/sda3 is identified by OS-prober as Windows recovery environment, recovery partition is not identified at all
Ubuntu boots fine, Windows stops with blinking cursor in the upper left corner right after grub before entering boot process.
grub-2 is correctly installed on /dev/sda.
I tried uninstall purge and reinstall with correct settings, same behaviour. Everything worked fine on Karmic, 9.10 (64bit)

On my old laptop (XP pro 32bit, 10.04 Lucid 32 bit) everytjhings works and boots fine.

Greetings Michael

[Erik Auerswald (auerswal)]

I see the misidentification problem on a Lenovo ThinkPad T61.

The Windows Recovery Partition is identified as Windows Vista, the Windows Vista Partition as Windows Recovery Environment.

[bdoe (bdoe-att)]

I had installed one of the later beta releases of Ubuntu 10.04 (latest version available in early April), and my Vista partition was identified correctly. Then, just yesterday, I finally noticed Update Manager trying to get my attention, so I updated. Now, my Vista partition is mis-identified as "Windows Recovery Environment". I do not have a recovery partition on this laptop. My boot priority also got changed (I had it set to boot Windows by default).

I like Grub2, but it really should allow me to manually make changes without having to rely on obscure scripts to autodetect things and lock me out of the process. That just isn't the Linux way. Menu.lst really needs to come back.

[bdoe (bdoe-att)]

I have a fix for the problem. This affects those with Windows Vista partitions being mis-identified as "Windows Recovery Environment". The problem is that os-prober is using /usr/lib/os-probes/mounted/20microsoft to read specific attributes of the identified Windows drive. One of these attributes is checking for "W.i.n.d.o.w.s. .R.e.c.o.v.e.r.y. .E.n.v.i.r.o.n.m.e.n.t" within C:\boot\BCD. In Vista and Windows 7, the BCD file will have that entry. Windows 7 users avoid this problem because os-prober checks for "W.i.n.d.o.w.s. .7" first, then checks for Windows Recovery Environment. It doesn't even look for a Vista entry, it just assumes the drive is a Vista drive if all other checks fail. The solution, therefore, is to specifically check for "W.i.n.d.o.w.s. .V.i.s.t.a" *before* checking for Windows Recovery Environment, and after checking for Windows 7. Ideally, the presence of "Windows Recovery Environment" should be the last thing checked.

I don't know if this is the ultimate solution, and I don't have an actual Windows Recovery partition to check this against, but the attached file works for me. Simply replace /usr/lib/os-probes/mounted/20microsoft with the attached file and re-run update-grub.?field.comment=I have a fix for the problem. This affects those with Windows Vista partitions being mis-identified as "Windows Recovery Environment". The problem is that os-prober is using /usr/lib/os-probes/mounted/20microsoft to read specific attributes of the identified Windows drive. One of these attributes is checking for "W.i.n.d.o.w.s. .R.e.c.o.v.e.r.y. .E.n.v.i.r.o.n.m.e.n.t" within C:\boot\BCD. In Vista and Windows 7, the BCD file will have that entry. Windows 7 users avoid this problem because os-prober checks for "W.i.n.d.o.w.s. .7" first, then checks for Windows Recovery Environment. It doesn't even look for a Vista entry, it just assumes the drive is a Vista drive if all other checks fail. The solution, therefore, is to specifically check for "W.i.n.d.o.w.s. .V.i.s.t.a" *before* checking for Windows Recovery Environment, and after checking for Windows 7. Ideally, the presence of "Windows Recovery Environment" should be the last thing checked.

I don't know if this is the ultimate solution, and I don't have an actual Windows Recovery partition to check this against, but the attached file works for me. Simply replace /usr/lib/os-probes/mounted/20microsoft with the attached file and re-run update-grub.

[Bouke Bunnik (bosyber)]

Brian Doe, thanks for your attempt at fixing this. I suffer from this issue, and I do have a recovery partition (well, suffer: I had to tell my wife not to boot Vista but the recovery partition instead). Your modification solves the misidentification of the Windows Vista partition, so in that sense it is an improvement.

But I now realize there were two problems in identification: the recovery partition is misidentified as Windows Vista. And this is not solved. I know that my 1st partition is the recovery, while sda2 is the Vista partition.

I believe that the recovery partition is not recognized, but just falling through to the default of Vista. Is the only way I can check that to install the hivex stuff mentioned above?

[Bouke Bunnik (bosyber)]

RIght, now that Brian Doe provided a fix for the Vista identification, I decided to see if my believe about the recovery partition not being matched and thus being detected as the default of Windows Vista was correct. Following the insructions from comment #7, I installed hivax, and checked the BCD record for those partitions.

I am sorry, I should have done that before, as it makes rather clear that the recovery partition was indeed not matched, and why:
./probe_bcd /dev/sda1 $bcd
Windows Setup
./probe_bcd /dev/sda2 $bcd
Microsoft Windows Vista

I tested that by adding another elif statement to the 20microsoft file by bdoe:
+ elif grep -qs "W.i.n.d.o.w.s. .S.e.t.u.p" "$2/$boot/$bcd"; then
+ long="Windows Recovery Environment (loader)"
 else
   long="Windows Vista (loader)"
  fi

Running update-grub with the attached file gives me:
Found memtest86+ image: /boot/memtest86+.bin
Found Windows Recovery Environment (loader) on /dev/sda1
Found Windows Vista (loader) on /dev/sda2

which is the correct result.

It might be that other manufacturers give the recovery partition another BCD record, so I am not claiming this will fix all, but I do think this is the right fix. Perhaps there is an easier way to record all the different names that might be used, and possibly it is a good idea to add hivex to the distribution in the future, to be able to diagnose this better in the future.

For Lucid it seems that just updating the required matching list seems like the easiest solution, but that requires all the possible matches to be identified.

Thanks Brian Doe for inspiring me to finally check this, and thanks TJ for the hivex package and the shell script. Sorry I did not try it before.

[Bouke Bunnik (bosyber)]

See the attached patch, combining the changes of bdoe and my added match for the recovery partition "Windows Setup", applies to version in Ubuntu Lucid/10.04 (maybe it is different from others because I have a Dutch version of Vista; Acer TravelMate 6292).

[Walter Garcia-Fontes (walter-garcia)]

I've just been hit by this bug when installing Ubuntu Lucid 10.04 from the alternate ISO image (64 bits) in a Dell Lattitude E6410. It came with three primary partitions. Dell Utilities in sda1, Windows 7 Recovery in sda2 and Windows 7 in sda3. I created an extended partition and install Ubuntu there.

Grub shows now all the Ubuntu starting options correctly, but the only extra option that is shown is Windows 7 and wants to star from sda2. It doesn't show the actual Windows 7 partition nor the Dell Utilities partition.

In an older Dell Latitude that I have grub legacy had identified all the partitions correctly.

[Walter Garcia-Fontes (walter-garcia)]

On my comment #23, what actually happened with the curren version of Grub OS Prober is that it identifiied correctly the
Windows 7 partition, which is at sda2, but it does not see the Dell Utility Partition at sda1.

[Jérôme De Greef (jdegreef)]

This is not a Ubuntu specific bug. Debian sid/unstable has the exact same issue.
See : http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589676 (patch included).

I think this should be fixed upstream.

[Anisse Astier (anisse)]

Just applied Bouke's modification, and it worked perfectly, recognizing a Recovery intead of mislabeling it "Windows Vista".

This should indeed be integrated upstream.

[Jérôme De Greef (jdegreef)]

I can confirm it's fixed... until Microsoft or Windows distributors call
their recovery partition another name.
It's a temporary fix.

BTW, it's not very fair to not cite me in the changelog as you've used part
(if not all) of my patch.

Jérôme

[Daniel Bair (danielbair)]

And not fixed for me! Unless the solution of comment #7 is implemented os-prober will not identify Recovery and Vista partitions correctly.

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows Vista (loader) (on /dev/sda2)" {
        insmod ntfs
        set root='(hd0,2)'
        search --no-floppy --fs-uuid --set 1a80d10f80d0f26d
        chainloader +1
}
menuentry "Windows Vista (loader) (on /dev/sda3)" {
        insmod ntfs
        set root='(hd0,3)'
        search --no-floppy --fs-uuid --set 423c6c953c6c862f
        chainloader +1
}
### END /etc/grub.d/30_os-prober ###

[Jérôme De Greef (jdegreef)]

God, as I said it was only a temporary fix and we already face a case where
it doens't work.

What's the BCD of your recovery partition ?

And btw what's comment #7 ? I don't see any number identifying the comments
here.

I assume this is not easy to fix as we depend on the way recovery partition
are named and this is up to the distributor to choose a name (Windows Setup,
Windows Recovery, Windows Recover Initial Setup, Windows Help It's Crashed,
whatever, you name it). Were're trying to deal with an os that as no
standard and that's not easy.

A solution would be to identify Windows 95, 200x, XP, Vista, Seven and put
anything else as revovery or unindentify (fat, fat32, htfs) but that will
only work until the next version of Windows, grrrrr.

[Launchpad Janitor (janitor)]

This bug was fixed in the package os-prober - 1.40

---------------
os-prober (1.40) unstable; urgency=low

  [ Christian Perrier ]
  * Fix Windows Vista and Windows Recovery Environment partitions
    recognition. (Thanks, Bouke Bunnik)
    Closes: #589676, LP: #476625
  * Allow recognition of recent MINIX installations.
    Thanks to Feiran Zheng
    Closes: #592924

  [ Colin Watson ]
  * Improve error message when /sys/block is missing.
  * os-prober doesn't know how to probe other OSes on non-Linux kernels.
    For now, just exit quietly rather than confusing people (closes:
    #567953).
  * Ignore active swap partitions (thanks, Alex Owen; see #417407).
  * Refactor linux_mount_boot to look up labels and UUIDs using blkid or
    /dev/disk/by-*/ rather than relying on mount being smart enough. This
    removes some horrible code that executes mount from /target.
  * Set partitions read-only before mounting them (based on a patch by Alex
    Owen; closes: #417407, #556739, #599203).
 -- Ubuntu Archive Auto-Sync <email address hidden> Fri, 12 Nov 2010 10:56:34 +0000

[Daniel Bair (danielbair)]

Ah, sorry, comment #7 is on the other bug report linked to this bug
report in launchpad.
Here is a quick link...
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/7
And I can confirm that using the hive method is the only way that
reliably works for me.
The reason why is in comment #4...
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/4

-Daniel

On Wed, Nov 10, 2010 at 6:20 PM, Jérôme De Greef
<email address hidden> wrote:
> God, as I said it was only a temporary fix and we already face a case where
> it doens't work.
>
> What's the BCD of your recovery partition ?
>
> And btw what's comment #7 ? I don't see any number identifying the comments
> here.
>
> I assume this is not easy to fix as we depend on the way recovery partition
> are named and this is up to the distributor to choose a name (Windows Setup,
> Windows Recovery, Windows Recover Initial Setup, Windows Help It's Crashed,
> whatever, you name it). Were're trying to deal with an os that as no
> standard and that's not easy.
>
> A solution would be to identify Windows 95, 200x, XP, Vista, Seven  and put
> anything else as revovery or unindentify (fat, fat32, htfs) but that will
> only work until the next version of Windows, grrrrr.
>
> --
> Grub2 affected when os-prober incorrectly identifies Windows and Recovery partitions
> https://bugs.launchpad.net/bugs/476625
> You received this bug notification because you are a direct subscriber
> of the bug.
>

[Daniel Bair (danielbair)]

This is the MAIN problem... assumption #2
2. The last entry in the /Boot/BCD file is related to the partition being probed

This is a false assumption in the case of my Toshiba Satellite system
and for many others also.

-Daniel

On Fri, Nov 12, 2010 at 7:37 AM, Daniel Bair <email address hidden> wrote:
> Ah, sorry, comment #7 is on the other bug report linked to this bug
> report in launchpad.
> Here is a quick link...
> https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/7
> And I can confirm that using the hive method is the only way that
> reliably works for me.
> The reason why is in comment #4...
> https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/4
>
>
> -Daniel
>
>
> On Wed, Nov 10, 2010 at 6:20 PM, Jérôme De Greef
> <email address hidden> wrote:
>> God, as I said it was only a temporary fix and we already face a case where
>> it doens't work.
>>
>> What's the BCD of your recovery partition ?
>>
>> And btw what's comment #7 ? I don't see any number identifying the comments
>> here.
>>
>> I assume this is not easy to fix as we depend on the way recovery partition
>> are named and this is up to the distributor to choose a name (Windows Setup,
>> Windows Recovery, Windows Recover Initial Setup, Windows Help It's Crashed,
>> whatever, you name it). Were're trying to deal with an os that as no
>> standard and that's not easy.
>>
>> A solution would be to identify Windows 95, 200x, XP, Vista, Seven  and put
>> anything else as revovery or unindentify (fat, fat32, htfs) but that will
>> only work until the next version of Windows, grrrrr.
>>
>> --
>> Grub2 affected when os-prober incorrectly identifies Windows and Recovery partitions
>> https://bugs.launchpad.net/bugs/476625
>> You received this bug notification because you are a direct subscriber
>> of the bug.
>>
>

[Jérôme De Greef (jdegreef)]

I complately agree with you.
I've used the hive method to clearly identify my windows partitions.
I fact we should stop to "hardcode" the partition names in
/usr/lib/os-probes/mounted/20microsoft.
Instead we should use the hive method and use it's result to identify
windows partitions. It's not a big deal I think except that we should have
everything (script, library,...) available at installation time.
As I said earlier this bug affects all distributions using os-prober. As
Debian is on the path to realease their next stable for me it's a critical
bug as it will confuse all user who don't have a big knowledge of linux
dists.

2010/11/12 Daniel Bair <email address hidden>

> This is the MAIN problem... assumption #2
> 2. The last entry in the /Boot/BCD file is related to the partition being
> probed
>
> This is a false assumption in the case of my Toshiba Satellite system
> and for many others also.
>
> -Daniel
>
>
> On Fri, Nov 12, 2010 at 7:37 AM, Daniel Bair <email address hidden>
> wrote:
> > Ah, sorry, comment #7 is on the other bug report linked to this bug
> > report in launchpad.
> > Here is a quick link...
> > https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/7
> > And I can confirm that using the hive method is the only way that
> > reliably works for me.
> > The reason why is in comment #4...
> > https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/476625/comments/4
> >
> >
> > -Daniel
> >
> >
> > On Wed, Nov 10, 2010 at 6:20 PM, Jérôme De Greef
> > <email address hidden> wrote:
> >> God, as I said it was only a temporary fix and we already face a case
> where
> >> it doens't work.
> >>
> >> What's the BCD of your recovery partition ?
> >>
> >> And btw what's comment #7 ? I don't see any number identifying the
> comments
> >> here.
> >>
> >> I assume this is not easy to fix as we depend on the way recovery
> partition
> >> are named and this is up to the distributor to choose a name (Windows
> Setup,
> >> Windows Recovery, Windows Recover Initial Setup, Windows Help It's
> Crashed,
> >> whatever, you name it). Were're trying to deal with an os that as no
> >> standard and that's not easy.
> >>
> >> A solution would be to identify Windows 95, 200x, XP, Vista, Seven and
> put
> >> anything else as revovery or unindentify (fat, fat32, htfs) but that
> will
> >> only work until the next version of Windows, grrrrr.
> >>
> >> --
> >> Grub2 affected when os-prober incorrectly identifies Windows and
> Recovery partitions
> >> https://bugs.launchpad.net/bugs/476625
> >> You received this bug notification because you are a direct subscriber
> >> of the bug.
> >>
> >
>
> --
> Grub2 affected when os-prober incorrectly identifies Windows and Recovery
> partitions
> https://bugs.launchpad.net/bugs/476625
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “grub2” package in Ubuntu: Invalid
> Status in “os-prober” package in Ubuntu: Fix Released
>
> Bug description:
> Binary package hint: grub
>
> The grub os-prober mis-identifies the Compaq recovery partition as Windows
> XP and the Windows XP partition as "ubuntu linux". Choosing Windows XP from
> the grub boot me...

[delance (olivier-delance)]

Same as https://launchpad.net/~intuitivenipple (TJ).

delance@delance-desktop:~$ sudo fdisk -l

Périphérique Amorce Début Fin Blocs Id Système
/dev/sda1 1 1567 12586896 27 Inconnu
/dev/sda2 * 1568 39238 302592307+ 7 HPFS/NTFS
/dev/sda3 39239 71109 256003807+ 83 Linux
/dev/sda4 71110 77825 53946270 83 Linux

delance@delance-desktop:~$ sudo os-prober
/dev/sda1:Windows Vista (loader):Windows:chain
/dev/sda2:Windows Recovery Environment (loader):Windows1:chain

In fact, recovery is first one and Vista second one.

[Jérôme De Greef (jdegreef)]

Can you please use TJ method using hives (BCD) to see what it gives on your
rescue partition (/dev/sda1) ?
Thanx

2010/11/13 delance <email address hidden>

> Same as https://launchpad.net/~intuitivenipple<https://launchpad.net/%7Eintuitivenipple>(TJ).
>
> delance@delance-desktop:~$ sudo fdisk -l
>
> Périphérique Amorce Début Fin Blocs Id Système
> /dev/sda1 1 1567 12586896 27 Inconnu
> /dev/sda2 * 1568 39238 302592307+ 7 HPFS/NTFS
> /dev/sda3 39239 71109 256003807+ 83 Linux
> /dev/sda4 71110 77825 53946270 83 Linux
>
> delance@delance-desktop:~$ sudo os-prober
> /dev/sda1:Windows Vista (loader):Windows:chain
> /dev/sda2:Windows Recovery Environment (loader):Windows1:chain
>
> In fact, recovery is first one and Vista second one.
>
> --
> Grub2 affected when os-prober incorrectly identifies Windows and Recovery
> partitions
> https://bugs.launchpad.net/bugs/476625
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “grub2” package in Ubuntu: Invalid
> Status in “os-prober” package in Ubuntu: Fix Released
>
> Bug description:
> Binary package hint: grub
>
> The grub os-prober mis-identifies the Compaq recovery partition as Windows
> XP and the Windows XP partition as "ubuntu linux". Choosing Windows XP from
> the grub boot menu runs the Compaq recovery program - a BAD idea. Choosing
> ubuntu linux on the other hand launches Windows XP correctly (or at least it
> did before today's update!). Substantiating information below:
>
> Grub was updated 6Nov 10:30AM but that doesn't change the behavior from
> before the update.
>
> <<< Output from fdisk -l with added comment identifying partition usage >>>
>
> Device Boot Start End Blocks Id System
> /dev/sda1 1 617 4664488+ b W95 FAT32
> Compaq recovery partition
> /dev/sda2 * 618 5372 35947800 7 HPFS/NTFS
> Windows XP
> /dev/sda3 5373 7269 14341320 83 Linux
> Ubuntu /
> /dev/sda4 7270 20671 101319120 f W95 Ext'd (LBA)
> /dev/sda5 7270 7404 1020568+ 82 Linux swap /
> Solaris /swap
> /dev/sda6 7405 20671 100298488+ 83 Linux
> /home
>
> <<< Partial content of /boot/grub/grub.cfg showing relevant sections >>>
>
> ### BEGIN /etc/grub.d/30_os-prober ###
> menuentry "Windows NT/2000/XP (on /dev/sda1)" {
> insmod fat
> set root=(hd0,1)
> search --no-floppy --fs-uuid --set 4452-51a8
> drivemap -s (hd0) ${root}
> chainloader +1
> }
> menuentry "ubuntu Linux (on /dev/sda2)" {
> insmod ntfs
> set root=(hd0,2)
> search --no-floppy --fs-uuid --set 3c78b47778b4318e
> drivemap -s (hd0) ${root}
> chainloader +1
> }
> ### END /etc/grub.d/30_os-prober ###
>
> <<< Partial list of grub.cfg generated by grub before today's update >>>
>
> ### BEGIN /etc/grub.d/30_os-prober ###
> menuentry "Windows NT/2000/XP (on...

[delance (olivier-delance)]

Packard Bell -
TJ#4
I get from 20microsoft:

     /dev/sda1 /media/_OEMBP fuseblk
/dev/sda1:Windows Vista (loader):Windows:chain
     /dev/sda2 /windows fuseblk
/dev/sda2:Windows Recovery Environment (loader):Windows1:chain

File systen is named _OEMBP.
In boot/BCD file of my recovery partition, I see any readable string meaningful.
Boot directories of Vista and recovery partitions have same list of files.
In my recovery partition, I have a "recovery" folder.

[Jérôme De Greef (jdegreef)]

2010/11/21 delance <email address hidden>

> Packard Bell -
> TJ#4
> I get from 20microsoft:
>
> /dev/sda1 /media/_OEMBP fuseblk
> /dev/sda1:Windows Vista (loader):Windows:chain
> /dev/sda2 /windows fuseblk
> /dev/sda2:Windows Recovery Environment (loader):Windows1:chain
>
> Is this correct or is /dev/sda1 your recovery partition and /dev/sda2 your
Vista (loader) partition ?

>
>

[delance (olivier-delance)]

It is not correct. sda1 is a small recovery partition and sda2 Vista partition.

[delance (olivier-delance)]

I installed os-prober 1.40 and now I get:

delance@delance-desktop:~$ sudo os-prober
[sudo] password for delance:
/dev/sda1:Windows Vista (loader):Windows:chain
/dev/sda2:Windows Vista (loader):Windows1:chain

So sda2 is not seen as a recovery partition, which is better.
But sda1 is not yet seen as a recovery partition.
I see a fix is released. What should I do ? Raise a new bug or reopen this one ?

[Jérôme De Greef (jdegreef)]

I'd reopen this one as it's obviously not fixed.
If I have time, I'll try to implement a solution using hives and BCD... if I
have time... I'm actually not in charge of os-prober but it seems that no
one care about fixing this for once and for all. This bug is more than one
year old now...

2010/11/23 delance <email address hidden>

> I installed os-prober 1.40 and now I get:
>
> delance@delance-desktop:~$ sudo os-prober
> [sudo] password for delance:
> /dev/sda1:Windows Vista (loader):Windows:chain
> /dev/sda2:Windows Vista (loader):Windows1:chain
>
> So sda2 is not seen as a recovery partition, which is better.
> But sda1 is not yet seen as a recovery partition.
> I see a fix is released. What should I do ? Raise a new bug or reopen this
> one ?
>
> --
> Grub2 affected when os-prober incorrectly identifies Windows and Recovery
> partitions
> https://bugs.launchpad.net/bugs/476625
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “grub2” package in Ubuntu: Invalid
> Status in “os-prober” package in Ubuntu: Fix Released
>
> Bug description:
> Binary package hint: grub
>
> The grub os-prober mis-identifies the Compaq recovery partition as Windows
> XP and the Windows XP partition as "ubuntu linux". Choosing Windows XP from
> the grub boot menu runs the Compaq recovery program - a BAD idea. Choosing
> ubuntu linux on the other hand launches Windows XP correctly (or at least it
> did before today's update!). Substantiating information below:
>
> Grub was updated 6Nov 10:30AM but that doesn't change the behavior from
> before the update.
>
> <<< Output from fdisk -l with added comment identifying partition usage >>>
>
> Device Boot Start End Blocks Id System
> /dev/sda1 1 617 4664488+ b W95 FAT32
> Compaq recovery partition
> /dev/sda2 * 618 5372 35947800 7 HPFS/NTFS
> Windows XP
> /dev/sda3 5373 7269 14341320 83 Linux
> Ubuntu /
> /dev/sda4 7270 20671 101319120 f W95 Ext'd (LBA)
> /dev/sda5 7270 7404 1020568+ 82 Linux swap /
> Solaris /swap
> /dev/sda6 7405 20671 100298488+ 83 Linux
> /home
>
> <<< Partial content of /boot/grub/grub.cfg showing relevant sections >>>
>
> ### BEGIN /etc/grub.d/30_os-prober ###
> menuentry "Windows NT/2000/XP (on /dev/sda1)" {
> insmod fat
> set root=(hd0,1)
> search --no-floppy --fs-uuid --set 4452-51a8
> drivemap -s (hd0) ${root}
> chainloader +1
> }
> menuentry "ubuntu Linux (on /dev/sda2)" {
> insmod ntfs
> set root=(hd0,2)
> search --no-floppy --fs-uuid --set 3c78b47778b4318e
> drivemap -s (hd0) ${root}
> chainloader +1
> }
> ### END /etc/grub.d/30_os-prober ###
>
> <<< Partial list of grub.cfg generated by grub before today's update >>>
>
> ### BEGIN /etc/grub.d/30_os-prober ###
> menuentry "Windows NT/2000/XP (on /dev/sda1)" {
> insmod fat
> set root=(hd0,1)
> search --no-flopp...

[Jonathan (jonathan-winterflood)]

I also have the same problem on a Packard Bell with Windows Vista, having just installed Ubuntu 10.10

### BEGIN /etc/grub.d/30_os-prober ###
menuentry "Windows Vista (loader) (on /dev/sda1)" {
    insmod part_msdos
    insmod ntfs
    set root='(hd0,msdos1)'
    search --no-floppy --fs-uuid --set 64bce3a1bce36bcc
    chainloader +1
}
menuentry "Windows Recovery Environment (loader) (on /dev/sda2)" {
    insmod part_msdos
    insmod ntfs
    set root='(hd0,msdos2)'
    search --no-floppy --fs-uuid --set bc3c53c13c5374fc
    drivemap -s (hd0) ${root}
    chainloader +1
}
### END /etc/grub.d/30_os-prober ###

In reality, Vista is on sda2, Recovery is on sda1

[Tom (tom6)]

Hi :)

I have helped people with the work-around a couple of times in the last couple of months. Since it affects a Windows system and Windows is often quirky it had not occurred to me to post a bug-report. Surely everyone is moving away from Windows anyway ?? (sry, irony)

I don't know which versions of os-prober were being used and it will be tricky for me to find the relevant threads. I think it affected fresh installs of Vista (which seems insane. i mean, why re-install Vista!?!?) and maybe Win7 too although i am not certain.

Sorry this post is so unhelpful! I will try to remember to get more detail!
Many thanks and regards from
Tom :)

[delance (olivier-delance)]

@Tom: I never reinstall Vista on my Packard Bell. For this brand, it seems, as I'm the second one with this issue (see post#41) that the way Vista and recovery is installed differ from usual way. Maverick CD (and current one from depot) version of os-prober is 1.39.

[Bouke Bunnik (bosyber)]

#43 - delance, that version of os-prober is from before the fix, explaining why you get the problem. A bit sad that this issue wasn't fixed before releasing the Maverick CD.

Jérôme De Greef, #40 Thanks for I think that hivex is a good way to check for what entries are present in the BCBD, but do you have a good suggestion for how to use it to get reliable os-prober entries?

The way I see it, os-prober's problem is that it is depending on knowing the right names for the recovery environment and for the "real" windows patitions on all machines to work perfectly - as we have seen, with each new release a new name is added for the latter, and the former, while often named "Windows Recovery Environment" is also know by at least one other name (my "Windows Setup"), and possibly more.

Even using hivex (and I am not entirely sure it is good to have to depend on that for correct grub installation ...) it will still be a matter of knowing the correct names at install time. It seems to me hivex here would only be able to tell that there are not previously encountered, and thus unclassified, names present.

There is one other thing that might help: heuristics, in this case: IF there are two windows partitions, and one is the recovery partition, will almost always be the 1st. So maybe using that as a default case might help misidentification.

If there are two different genuine windows partitions, without a recovery partition, this might lead again to a wrong naming. But, in that case, it is very unlikely a factory install, meaning that the person who installed the machine knows what he/she is doing, and might be helped by a notification from the 1st time install of grub mentioning the issue with os-prober and what can be done to sort it. Less harmful anyway than accidentally starting a recovery (lead to losing the linux root for me - good to have backups, it lead me, amongst other things to try out natty).

I don't really see a good way to get it right out of the box, always. I hope that is just me though, and I agree that it would be good to have a pre-install hivex check to see if there are "new" entries in BCD, and warn.

[Chow Loong Jin (hyperair)]

Here's a debdiff for a SRU to Maverick.

[JW (jw-00000)]

I experienced the same problem on my Acer AS3810T, on Maverick. However, it seems to be fixed in os-prober version >= 1.40 (see last message in the thread for this bug on debian.org, [1]).

I installed the .deb for os-prober (version 1.42) from natty, found here[2], and ran the commands:
$ sudo os-prober
$ sudo update-grub

Now, everything works fine :-)
I hope os-prober v1.42 can be backported to maverick.

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589676
[2] http://packages.ubuntu.com/natty/os-prober