update-grub fails in the face of overlayroot

[Mike Purvis]

Info on overlayroot: http://blog.dustinkirkland.com/2012/08/introducing-overlayroot-overlayfs.html

Mount configuration for an overlayroot-configured box:

$ mount
overlayroot on / type overlayfs (rw,errors=remount-ro)
/dev/sda5 on /media/root-ro type ext3 (ro,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered)
/dev/sda6 on /media/root-rw type ext3 (rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered)
/dev/sda1 on /boot type ext3 (rw)

When attempting to update-grub:

$ sudo update-grub
[sudo] password for administrator:
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).

The issue comes about in this line of the /usr/sbin/grub-mkconfig script:

GRUB_DEVICE="`${grub_probe} --target=device /`"

In the short term, I'm hacking that script to pass in a value for GRUB_DEVICE, but the ideal would be to have the grub-probe binary detect when an overlayfs is / and return the device which is the bottom of the overlay.

[Mike Friedrich (mfriedrich74)]

This affects all releases, not just trusty. e.g. xenial and bionic

[Matthias Kruzenski (m4dm4x1337)]

WORKING SOLUTION
================

After encountering this problem, I developed a very simple solution:

nano /usr/sbin/grub-probe-overlay

-----[BEGIN]--------------------------------------------------------------
#!/bin/sh

ARGS=

for ARG in $@; do
 if [ "${ARG%%[!/]*}" = "/" ]; then

  OPTIONS=

  if FSTYPE=$( findmnt -n -o FSTYPE "$ARG" ); then
   if [ "$FSTYPE" = "overlay" ]; then
    OPTIONS=$(findmnt -n -o OPTIONS "$ARG")
   fi
  elif FSTYPE=$( findmnt -n -o FSTYPE / ); then
   if [ "$FSTYPE" = "overlay" ]; then
    OPTIONS=$(findmnt -n -o OPTIONS /)
   fi
  fi

  if LOWERDIR=$(echo "$OPTIONS" | grep -m 1 -oP 'lowerdir=\K[^,]+'); then
# echo "[DEBUG] Overlay filesystem detected \${ARG} --> \${LOWERDIR}\${ARG%*/}" 1>&2
   ARG="${LOWERDIR}${ARG%*/}"
  fi
 fi
 ARGS="$ARGS $ARG"
done

grub-probe-default $ARGS

exit $?
-----[END]----------------------------------------------------------------

mv /usr/sbin/grub-probe /usr/sbin/grub-probe-default

ln -s /usr/sbin/grub-probe-overlay /usr/sbin/grub-probe

OPTIONAL STEP:
==============

apt-mark hold grub-pc

NOTE: package name can also be "grub-efi-amd64" or "grub-efi-ia32" or something else, depending on what you use

PROS
====
- works perfectly with overlay filesystem
- shows a general solution (maybe someone can use it to fix the grub-probe source code)
- no patching of grub scripts required (some people tried patches as solution, but grub-probe is used in many grub scripts --> patching nightmare)

CONS
====
- grub-probe must be renamed (for this reason, I recommend that you set the status of your grub package to "hold")

[Matthias Kruzenski (m4dm4x1337)]

Sorry, I forgot

chmod +x /usr/sbin/grub-probe-overlay

[Mike Rushton (leftyfb)]

This bug is still present in Ubuntu 16.04.5

[Atif Mahmood (atif1996)]

Bug exists in 18.04 as well.