Root filesystem (used by LXD) include device nodes

[Stéphane Graber]

It looks like some change in image generation has occurred and results in the rootfs tarballs and/or squashfs now containing device nodes.

So far, we've noticed:
 - /dev/console
 - /dev/full
 - /dev/mapper/control
 - /dev/null
 - /dev/ptmx
 - /dev/random
 - /dev/tty
 - /dev/urandom
 - /dev/zero

This causes issue with LXD, especially in the nested case as unprivileged container cannot (for security reason) create device nodes. The inclusion of those device nodes even in the non-nested case can be problematic as it could grant additional device access to containers above what we normally provide through LXD.

That last point is thankfully mitigated through the devices cgroup that we also configured in LXD, so I'm not filing this as a security issue, but I sure prefer the devices cgroup being there as a safety net rather than as the main security mechanism.

To validate a fix, you can attempt to unsquashfs the rootfs inside of a LXD container.
This should normally succeed with no error and an exit code of 0.

Instead, we're getting:
```
root@nesting:~# unsquashfs -n /var/snap/lxd/common/lxd/images/e3e1bd82cdc7fa1256cf2409dd8543630eefa1fca631ff0c78c0970babddc69f.rootfs
Parallel unsquashfs: Using 4 processors
36619 inodes (41808 blocks) to write

create_inode: failed to create character device squashfs-root/dev/console, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/full, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/mapper/control, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/null, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/ptmx, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/random, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/tty, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/urandom, because Operation not permitted
create_inode: failed to create character device squashfs-root/dev/zero, because Operation not permitted

created 31866 files
created 3728 directories
created 4632 symlinks
created 0 devices
created 0 fifos
created 0 sockets
root@nesting:~# echo $?
2
```

[Stéphane Graber (stgraber)]

This was reported to us at https://github.com/lxc/lxd/issues/10492

[Stéphane Graber (stgraber)]

After further digging, /dev has been populated for a long time (maybe forever?) but LXD has a workaround which parses the unsquashfs output (ugh) and ignores specific errors to accommodate that.

This particular hack stopped working recently causing this issue. The community images have always shipped with an empty /dev so those weren't impacted, causing the current situation.

[Stéphane Graber (stgraber)]

Submitted https://github.com/lxc/lxd/pull/10495 which fixes our previous workaround.

[Thomas Bechtold (toabctl)]

I still see device nodes available in our shipped squashfs files (eg. https://cloud-images.ubuntu.com/mantic/20230920/mantic-server-cloudimg-amd64.squashfs ).