snapper rollback does not work on ubuntu for btrfs

[Shuhao]

snapper rollback for root volumes have no effect. After calling snapper rollback, the default is being set correctly as seen via btrfs subvolume get-default. However, ubuntu doesn't seem to boot into that snapshot.

I see an upstream report: https://github.com/openSUSE/snapper/issues/365

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: snapper 0.5.0-2
ProcVersionSignature: Ubuntu 4.13.0-17.20-generic 4.13.8
Uname: Linux 4.13.0-17-generic x86_64
NonfreeKernelModules: zfs zunicode zavl zcommon znvpair
ApportVersion: 2.20.7-0ubuntu3.5
Architecture: amd64
CurrentDesktop: MATE
Date: Sat Nov 25 19:22:33 2017
InstallationDate: Installed on 2017-11-10 (15 days ago)
InstallationMedia: Ubuntu-MATE 17.10 "Artful Aardvark" - Release amd64 (20171018)
ProcEnviron:
 LANGUAGE=en_CA:en
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_CA.UTF-8
 SHELL=/bin/zsh
SourcePackage: snapper
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.default.snapper: [modified]
mtime.conffile..etc.default.snapper: 2017-11-21T00:16:24.063561

[Shuhao (shuhao)]

So I think I know why this doesn't work under Ubuntu and it's related to the mount options Ubuntu sets by default for btrfs for a new installation.

By default, Ubuntu installs systems with btrfs into a subvolume called @ and the home directory into a subvolume called @home. It then proceeds to add the mount options of subvol=@ to /etc/fstab for the root volume's mount. This is all great until you want to revert via snapper.

Before going further, I want to clarify that, afaik, snapshots are just subvolumes for BTRFS. For snapper, the subvolume has a path of /.snapshots/<int>/snapshot. Since this is relative to the rootfs mounted, which is @, I believe it's actually @/.snapshots/<int>/snapshot.

snapper's rollback works by using the default subvolume feature for a filesystem. This means if you mount that file system (mount /dev/sda1), it will mount that subvolume instead of the top-level file system.

As an example, when rolling back to @/.snapshots/10/snapshot, snapper creates a read-writable subvolume from snapshot #10 and sets that as the default subvolume for the filesystem instead of the usual /. Its design hopes that on the next boot of the system, it will boot from the newly cloned rw-able snapshot, as opposed to the original subvolume. Note that it does not alter @.

However, since ubuntu's fstab entry for the root volume has subvol=@ as the default flag, it will ALWAYS mount subvol=@ as / and never mount the default subvolume as /.

As a workaround, you could in theory:

1. Remove the subvol=@ options from /etc/fstab
2. Set the default subvolume for your filesystem to be @ (or whatever snapshot you're using).

I haven't tested this yet, but it _should_ fix this problem. Although this may be more preferable to be done by the ubuntu installer by default instead. However, doing that is a major change and might have other implications that I'm not aware of.

Additionally, the .snapshots folder under @ is not quite idea for rollbacks from a liveCD... I don't know what the optimal layout would be tho, but having it nested under @ means you can't do something like mv @/.snapshots/.../snapshot @ easily.

[Ahmed (akha666-f)]

I tested your workaround with Ubuntu 16.04 with no luck.
1- remove subvol=@
2- also add the same root ID with
UUID=ce4d5d13-5ec7-4b2f-b519-569e04729d1e /.snapshots btrfs subvol=@/.snapshots 0 1
as OpenSuSE fstab.
3- btrfs subvolume set-default (SNAP NUMBER) /
any help? I need to rollback as easy to do in OpenSuSE.

[Shuhao (shuhao)]

Interesting... I still can't easily test right now, but if you try to mount the device manually, possibly from an live disk (`mount /dev/sdXX /path`), does the right subvol get mounted?

[James Seiters (jseiters)]

I think I have found a way but it is more of a hack. When I remove:
rootflags=subvol=${rootsubvol}
from the line GRUB_CMDLINE_LINUX="rootflags=subvol=${rootsubvol} ${GRUB_CMDLINE_LINUX}" in the file /etc/grub.d/10_linux I can boot from the rw snapshot snapper makes when I issue the command:
sudo snapper rollback 38
You will have to do an update-grub before the snapper rollback command to write the new cfg file.
This method will most likely fail if an update replaces the 10_linux file and I don't know how safe it is considering it is not officially supported.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Javier Lopez (muniter)]

I've experienced this as well, thanks to @jseiters I was able to perform a rollback. But is definitely not a nice experience when urgently needing the rollback for the first time!

Hope It's fixed.

[baltic (bugsgenerator)]

the method above doesn't fully work, since even though grub doesn't pass the subvol= option to the kernel, it still tries to find kernel at full path, including the subvolume name.
e.g. in /boot/grub/grub.cfg there are lines like:
linux /@/boot/vmlinuz-5.13.0-20-generic

to make it work as easy as on OpenSuse i have made separate subvolume for /boot

1.
in /etc/grub.d/10_linux
i have removed (commented out) the lines:

# case x"$GRUB_FS" in
# xbtrfs)
# rootsubvol="`make_system_path_relative_to_its_root /`"
# rootsubvol="${rootsubvol#/}"
# if [ "x${rootsubvol}" != x ]; then
# GRUB_CMDLINE_LINUX="rootflags=subvol=${rootsubvol} ${GRUB_CMDLINE_LINUX}"
# fi;;
# xzfs)
# # We have a more specialized ZFS handler, with multiple system in 10_linux_zfs.
# if [ -e "`dirname $(readlink -f $0)`/10_linux_zfs" ]; then
# exit 0
# fi
# rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
# bootfs="`make_system_path_relative_to_its_root / | sed -e "s,@$,,"`"
# LINUX_ROOT_DEVICE="ZFS=${rpool}${bootfs%/}"
# ;;
# esac

Though i could have commented out only the btrfs related path, but i don't use zfs anyway

2.
Then mounted the btrfs root subvolume (subvolid=5 option) sudo mount -L sys -o subvolid=5 /mnt
and created @boot subvolume there, and copied (cp -a) the boot directory from my previous boot.

3.
created fstab record for the new boot:
LABEL=sys /boot btrfs defaults,subvol=@boot 0 2
and mounted it
mount -a
mount /boot/efi

4.
update-grub

Now works fine even on top of luks.
I have also created separate subvolume for /var/log but this is optional.