grub-probe takes snapshot LV instead of origin

[MegaBrutal]

grub-probe takes the snapshot of my root LV as if it was the legitimate root device.
This is an issue because in any case an update-grub / grub-mkconfig runs, my GRUB config will be rewritten so that the system would boot from the snapshot. This is really undesirable.

root@thinkpad:~# mount | grep " / "
/dev/mapper/thinkvg-rootlv on / type btrfs (rw,subvol=@)

root@thinkpad:~# lvs
  LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
  homelv thinkvg -wi-ao--- 64,00g
  rootlv thinkvg owi-aos-- 16,00g
  snap-rootlv thinkvg sri-a-s-- 16,00g rootlv 2,54
  swap0 thinkvg -wi-ao--- 2,00g
  swap1 thinkvg -wi-ao--- 2,00g

root@thinkpad:~# grub-probe --target=device /
/dev/mapper/thinkvg-snap--rootlv

The most apparent reason could be that obviously the UUIDs of the origin and the snapshot are the same.

root@thinkpad:~# blkid /dev/thinkvg/*rootlv
/dev/thinkvg/rootlv: LABEL="SystemRoot" UUID="dd6904ff-d187-43a8-ae18-478283f29d68" UUID_SUB="80b3d748-51e0-4a51-a021-55005b0ec434" TYPE="btrfs"
/dev/thinkvg/snap-rootlv: LABEL="SystemRoot" UUID="dd6904ff-d187-43a8-ae18-478283f29d68" UUID_SUB="80b3d748-51e0-4a51-a021-55005b0ec434" TYPE="btrfs"

However, strangely, if I hide the snapshot by deleting nodes (as I can't deactivate a snapshot without deactivating its origin too – lost nodes will come back after reboot, or sometimes can be recovered with "vgscan --mknodes"), grub-probe won't find any device:

root@thinkpad:~# rm /dev/thinkvg/snap-rootlv /dev/mapper/thinkvg-snap--rootlv /dev/dm-5
root@thinkpad:~# grub-probe --target=device /
grub-probe: error: cannot find a device for / (is /dev mounted?).

I'd expect grub-probe to find /dev/mapper/thinkvg-rootlv, and ignore the snapshot.

For me, it serves as an evidence that grub-probe totally ignores LVM, or at least makes no attempt to differentiate snapshots from origins, while LV snapshots are easily recognized from their "s" flag.

The same thing happened when I manually cloned the root LV as "backup-rootlv", though since it was created with simply dd-ing "rootlv" over, the LVs were not related on LVM level, thus grub-probe legitimately treated it as a stand-alone device that has the same file system as the one being mounted.

Still, it would be best is grub-probe found the device that is actually mounted, no matter what clones of the device exist. But at least it should certainly ignore LVM snapshots.

I experienced this bug on 2 systems: on Trusty Tahr and on Utopic Unicorn.

[Phillip Susi (psusi)]

Grub does ignore snapshots... what does cat /proc/mounts say?

[MegaBrutal (qbu6to)]

I made some experiments and could only reproduce the problem with btrfs file system.
And /proc/mounts says the root file system is the snapshot. This is rather strange.
Probably it's a btrfs bug and nothing to do with grub-probe.

To reproduce:
1. Create an LV, format it to btrfs.
2. Mount the LV, create subvolume '@' on the file system.
3. debootstrap a base system to the subvolume.
4. chroot to the subvolume, do whatever it takes to make it bootable (install kernel, lvm2, btrfs-tools).
5. Boot the new system.
6. Create an LVM-snapshot of the root LV.
7. 'grub-probe --target=device /' will tell you that your root device is the snapshot.

[Phillip Susi (psusi)]

So /proc/mounts lists the snapshot as the root... what does /proc/cmdline say?

[MegaBrutal (qbu6to)]

The kernel is not being rebooted meanwhile, so /proc/cmdline, so it still states the original device as root.
For example:
root=/dev/vg/rootlv rootflags=subvol=@ ro

[Phillip Susi (psusi)]

If /proc/mounts lists the wrong device, then that certainly is a kernel bug.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1391429

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.18 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.18-rc5-vivid/

[MegaBrutal (qbu6to)]

I've tested with 3.18.0-031800rc5-generic amd64. I experience the same.

[MegaBrutal (qbu6to)]

I've built a system to reproduce the bug. You can download the image and run it with KVM or other virtualization technology. Instructions are straightforward – if you start the VM, you'll know what to do, and you'll see what I was talking about.

http://undead.megabrutal.com/kvm-reproduce-1391429.img.xz

Download size: 113 MB; Unpacked image size: 2 GB.

[MegaBrutal (qbu6to)]

Reported upstream to BTRFS developers:
https://bugzilla.kernel.org/show_bug.cgi?id=89121