Activity log for bug #2003667

Date Who What changed Old value New value Message
2023-01-23 01:39:30 Tom Eastman bug added bug
2023-01-23 01:44:10 Tom Eastman description Snaps started being listed as `broken` after rebooting. After a lot of trouble, I discovered that the `squashfs` file that contains the snap was failing to be mounted, because the ZFS partition that it was on hadn't finished mounting. My hypothesis is that snapd `2.58` (installed as a snap) when *combined* with snap `2.57` (as installed by apt in Ubuntu 22.04) fails on ZFS systems because the unit file required that depends on ``` tak# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS" ``` ``` tak# snap --version snap 2.58 snapd 2.58 series 16 ubuntu 22.04 kernel 5.15.0-58-generic ``` ``` tak# snap list | grep snapd snapd 2.58 17950 latest/stable canonical** snapd ``` ``` tak# apt-cache policy snapd snapd: Installed: 2.57.5+22.04ubuntu0.1 Candidate: 2.57.5+22.04ubuntu0.1 Version table: *** 2.57.5+22.04ubuntu0.1 500 500 http://nz.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages 100 /var/lib/dpkg/status 2.55.3+22.04 500 500 http://nz.archive.ubuntu.com/ubuntu jammy/main amd64 Packages ``` Ubuntu 22.04 installs `2.57`, but then Snap updates itself with the snap for `2.58`. In this situation, the `.mount` file generated includes the clause `After=snapd.mounts-pre.target` but the actual unit file for `snapd.mounts-pre.target` is *not* installed, and isn't present in `/etc/systemd/system` (or wherever it's supposed to go, I'm not an expert at systemd) Unit files with the right configuration *do* exist in `/snap/snapd/` but they're not installed into a directory where systemd will process them: ``` tak# find /snap/snapd/current/ -name "*.target" /snap/snapd/current/lib/systemd/system/snapd.mounts-pre.target /snap/snapd/current/lib/systemd/system/snapd.mounts.target ``` So the fix for my system was to manually copy these files to `/etc/systemd/system/` I think, if the snap for `snapd` is going to be installed to "upgrade" it, it'll need to put these systemd files in the right place or it'll continue to fail for Ubuntu systems running on ZFS. Snaps started being listed as `broken` after rebooting. After a lot of trouble, I discovered that the `squashfs` file that contains the snap was failing to be mounted, because the ZFS partition that it was on hadn't finished mounting. My hypothesis is that snapd `2.58` (installed as a snap) when *combined* with snap `2.57` (as installed by apt in Ubuntu 22.04) fails on ZFS systems because the unit file required that ensures `zfs.mount.service` is read (`snapd.mounts-pre.target`) is never put in place by the *snap* for snapd `2.58`. ``` tak# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS" ``` ``` tak# snap --version snap 2.58 snapd 2.58 series 16 ubuntu 22.04 kernel 5.15.0-58-generic ``` ``` tak# snap list | grep snapd snapd 2.58 17950 latest/stable canonical** snapd ``` ``` tak# apt-cache policy snapd snapd:   Installed: 2.57.5+22.04ubuntu0.1   Candidate: 2.57.5+22.04ubuntu0.1   Version table:  *** 2.57.5+22.04ubuntu0.1 500         500 http://nz.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages         500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages         100 /var/lib/dpkg/status      2.55.3+22.04 500         500 http://nz.archive.ubuntu.com/ubuntu jammy/main amd64 Packages ``` Ubuntu 22.04 installs `2.57`, but then Snap updates itself with the snap for `2.58`. In this situation, the `.mount` file generated includes the clause `After=snapd.mounts-pre.target` but the actual unit file for `snapd.mounts-pre.target` is *not* installed, and isn't present in `/etc/systemd/system` (or wherever it's supposed to go, I'm not an expert at systemd) Unit files with the right configuration *do* exist in `/snap/snapd/` but they're not installed into a directory where systemd will process them: ``` tak# find /snap/snapd/current/ -name "*.target" /snap/snapd/current/lib/systemd/system/snapd.mounts-pre.target /snap/snapd/current/lib/systemd/system/snapd.mounts.target ``` So the fix for my system was to manually copy these files to `/etc/systemd/system/` I think, if the snap for `snapd` is going to be installed to "upgrade" it, it'll need to put these systemd files in the right place or it'll continue to fail for Ubuntu systems running on ZFS.