We're still struggling with this bug in our spread tests. When running spread tests for *craft applications, we occasionally see the following failure: + snap install lxd error: cannot perform the following tasks: - Run configure hook of "lxd" snap if present (run hook "configure": ----- cat: /proc/self/attr/current: Permission denied /snap/lxd/23680/snap/hooks/configure: 5: exec: aa-exec: Permission denied -----) The failure is occurring in clean VMs with a stock Ubuntu image (18.04, 20.04, and 22.04). The failure rate for these steps is very low - less than 1% of the time. In our github workflow, we run ~350 spread tests, so the failure of the overall work flow is around 10-15% (although some weeks it has been closer to 50%). The environment setup is brief. We apply the following steps to the image: apt-get install -y snapd snap install snapd snap wait system seed.loaded if [ "$SPREAD_SYSTEM" = "ubuntu-18.04-64" ] || [ "$SPREAD_SYSTEM" = "ubuntu-20.04-64" ]; then # Remove lxd and lxd-client deb packages as our implementation (pylxd) does not # nicely handle the snap and deb being installed at the same time. apt-get remove --purge --yes lxd lxd-client fi # Install and setup the lxd snap snap install lxd ------------------ Attempted solutions I tried the 3 primary suggestions from this thread (https://forum.snapcraft.io/t/snapped-lxd-has-stopped-working-aa-exec-permission-denied/2356) with no luck: 1. Running snap interfaces (now snap connections) This was suggested to check if the LXD snap is connected to lxd-support interface. This displays nothing, because the LXD snap is not installed yet. 2. Running snap install core20 and snap revert core20 This gives the error: ``` + snap revert core20 error: cannot revert "core20": no revision to revert to ``` 3. Running snap install core20 and snap refresh core20 This still produces the same exec: aa-exec: Permission denied error. I’ve captured the journal of a failure here (https://paste.ubuntu.com/p/XnyFRsvVfH/), with the failing section posted below: ``` oct142128-679423 audit[1763]: AVC apparmor="DENIED" operation="open" profile="snap.lxd.activate" name="/proc/1763/attr/current" pid=1763 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 oct142128-679423 audit[1764]: AVC apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0 oct142128-679423 audit[1764]: AVC apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0 oct142128-679423 kernel: kauditd_printk_skb: 4 callbacks suppressed oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:46): apparmor="DENIED" operation="open" profile="snap.lxd.activate" name="/proc/1763/attr/current" pid=1763 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:47): apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0 oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:48): apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0 oct142128-679423 lxd.activate[1744]: The LXD snap was unable to run aa-exec, this usually indicates a LXD sideload. oct142128-679423 lxd.activate[1744]: When sideloading, make sure to manually connect all interfaces. oct142128-679423 systemd[1]: snap.lxd.activate.service: Deactivated successfully. ```