os-prober leaves filesystems (lvm-thin, lvm snap) mounted

[mike]

If one happens to have by accident (or on purpose) a thick LVM snapshot, or a thin-LV with a copy of a system on it, update-grub reports
/sbin/grub-probe: error: unknown filesystem.
Found Ubuntu 22.04.1 LTS on /dev/mapper/vgxubuntu-s22.04
and
/sbin/grub-probe: error: disk `lvmid/X7waXv-rMDA-5fQA-aw6l-ei1I-8Gz3-pQl3lr/k3Ze1u-Mfwc-pxUt-7fXa-FjSZ-7IkX-HAxf3b' not found.
Found Ubuntu 22.04.1 LTS (22.04) on /dev/mapper/vgxubuntu-XR22.04.1U

and leaves the systems mounted rw on /var/lib/os-prober/mount.
On subsequent invocations update-grub spews out multiple
mdir: failed to remove '/var/lib/os-prober/mount': Device or resource busy

I'm pretty sure this also causes a new install to hang (though a power-reset seems to load the new system - not sure if it is complete or not.)

The missing umounts should be relatively easy to fix, but the thick snapshot error is particularly vague as most users (I think) would take the "unknown filesystem" to refer to the previous entry.

I have been playing with grub2-common-2.06-2ubuntu7 on xubuntu 22.04.1, though I am sure this has been around a while

[Julian Andres Klode (juliank)]

I've seen various issues with os-prober where it hangs when trying to mount snapshots. I thought we made it skip them all. Sigh.

I'm not sure it is missing unmounts vs having no success unmounting it as the mount point is kept busy by some other process.

[mike (mikeaa)]

I don't think it is os-prober, at least if I run os-prober directly it doesn't leave systems mounted, but it does return that that it found a system on the snapshots or thin-LVs.
It is only when the output of os-prober is processed by grub (30_os-prober) do the mounts get left.

I guess one could argue that os-prober should not report systems found on snapshots or thin-lvms or whatever other filesystem GRUB can't handle and that would fix the problem.

As it stands the mount (that is "being left") is being done by grub.

I was unable to compile the source for grub (apt-get source grub2-common) and I'm afraid I kinda got lost trying to figure out just where the problem is by reading the source.

I have gone a different route and replaced 30_os-prober (would be better if os-prober still reported boot partitions), so that I can boot from linear lV boot partitions and thin-lv system partitions.
But I still have the original stuff to experiment with if needed.

[mike (mikeaa)]

well Julian is right and I am wrong. I apologize I should have picked up on it quicker (I'm sick, too many drugs and not the good kind)

The culprit is linux-boot-prober which executes /usr/lib/linux-boot-probe/50mounted-tests which

set -e
...
if type grub-mount >/dev/null 2>&1 && \
   type grub-probe >/dev/null 2>&1 && \
   grub-mount "$partition" "$tmpmnt" 2>/dev/null; then
        mounted=1
        type="$(grub-probe -d "$partition" -t fs)"

grub-probe fails causing 50mounted-tests to quit with the partition mounted.

simply changing to
        type="$(grub-probe -d "$partition" -t fs || true)"

solves the mount problem, correctly(?) ignores a thin-partition, but adds bad entries for the thick snapshot (if it was accidental - it will boot into the base of the snapshot, not sure all that will happen if fstab and grub.cfg were edited)

changing it to
        type="$(grub-probe -d "$partition" -t fs || echo bad)"
and adding a case entry for bad to unmount and then quit would at least create a correct grub.cfg.

[Utkarsh Gupta (utkarsh)]

Ubuntu 22.10 (Kinetic Kudu) has reached end of life, so this bug will not be fixed for that specific release.