Comment 0 for bug 1967576

This build log provides an example of the problem: https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/ppc64el/s/systemd/20220331_190433_fdbbe@/log.gz.

During setup for tests-in-lxd, an attempt is made to purge snapd, but this fails:

[ ... ]
Purging configuration files for snapd (2.54.3+21.10.1ubuntu0.2) ...
Stopping snap.lxd.activate.service
Stopping unit snap.lxd.activate.service
Waiting until unit snap.lxd.activate.service is stopped [attempt 1]
snap.lxd.activate.service is stopped.
Removing snap.lxd.activate.service
Stopping snap.lxd.daemon.service
Stopping unit snap.lxd.daemon.service
Waiting until unit snap.lxd.daemon.service is stopped [attempt 1]
snap.lxd.daemon.service is stopped.
Removing snap.lxd.daemon.service
Stopping snap.lxd.user-daemon.service
Stopping unit snap.lxd.user-daemon.service
Waiting until unit snap.lxd.user-daemon.service is stopped [attempt 1]
snap.lxd.user-daemon.service is stopped.
Removing snap.lxd.user-daemon.service
Stopping snap-core20-1378.mount
Stopping unit snap-core20-1378.mount
Waiting until unit snap-core20-1378.mount is stopped [attempt 1]
snap-core20-1378.mount is stopped.
Removing snap core20 and revision 1378
Removing snap-core20-1378.mount
Stopping snap-lxd-22672.mount
Stopping unit snap-lxd-22672.mount
Waiting until unit snap-lxd-22672.mount is stopped [attempt 1]
snap-lxd-22672.mount is stopped.
Removing snap lxd and revision 22672
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/cpuinfo': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/meminfo': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/stat': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/uptime': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/diskstats': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/swaps': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/loadavg': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/proc/slabinfo': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/cpu0': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/cpuidle': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/uevent': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/svm': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/hotplug': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/offline': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/present': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/power': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/dscr_default': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/probe': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/smt': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/vulnerabilities': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/cpufreq': Operation not permitted
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/isolated': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/kernel_max': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/sys/devices/system/cpu/modalias': Function not implemented
rm: cannot remove '/var/snap/lxd/common/var/lib/lxcfs/cgroup': Input/output error
dpkg: error processing package snapd (--purge):
 installed snapd package post-removal script subprocess returned error exit status 1
[ ... ]

This appears to result in some snap .mount units being left over in /etc/systemd/system. These leftover mount units will now always fail because snapd etc. is no longer installed. Thus, the 'test_no_failed' test, which checks for any failed units after boot, will fail:

[ ... ]
======================================================================
FAIL: test_no_failed (__main__.ServicesTest)
No failed units
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.s5bwuE/build.2Vb/real-tree/debian/tests/boot-and-services", line 69, in test_no_failed
    self.assertEqual(failed, [])
AssertionError: Lists differ: ['snap-lxd-22672.mount loaded failed fai[119 chars]180'] != []

First list contains 2 additional elements.
First extra element 0:
'snap-lxd-22672.mount loaded failed failed Mount unit for lxd, revision 22672'

+ []
- ['snap-lxd-22672.mount loaded failed failed Mount unit for lxd, revision '
- '22672',
- 'snap-snapd-15180.mount loaded failed failed Mount unit for snapd, revision '
- '15180']

----------------------------------------------------------------------
Ran 23 tests in 3.771s

FAILED (failures=1, skipped=7)
autopkgtest [18:16:43]: test boot-and-services: -----------------------]
autopkgtest [18:16:44]: test boot-and-services: - - - - - - - - - - results - - - - - - - - - -
boot-and-services FAIL non-zero exit status 1
autopkgtest [18:16:44]: @@@@@@@@@@@@@@@@@@@@ summary
boot-and-services FAIL non-zero exit status 1
autopkgtest [18:16:52]: test tests-in-lxd: -----------------------]
autopkgtest [18:16:54]: test tests-in-lxd: - - - - - - - - - - results - - - - - - - - - -
tests-in-lxd FAIL non-zero exit status 1
[ ... ]

The underlying snapd bug appears to be related to bug 1903967 and bug 1950864. In the mean time, the test should be fixed so this does not block systemd autopktests.