non-obvious behaviour of path reconstruction with multiple choices after pivot_root and bind-mounts

[Zygmunt Krynicki]

The snap-confine program uses pivot_root and a some number of bind mounts to construct a mount namespace that looks, roughly, like this:

/snap/core/1234/ <- mounted squashfs over loopback device (bind mounted from below)
/var/lib/snapd/hostfs <- location of old root filesystem after pivot_root
/var/lib/snapd/hostfs/snap/core/1234/ <- the same squashfs as above (original location)

If a program is opened using open(2) with O_PATH from /snap/core/1234/<subdirectory> and the descriptor is carried across pivot_root and used with fexecve(3) then apparmor rule enforcing it will refer to /var/lib/snapd/hostfs/snap/core/1234/<subdirectory> rather than the (at least to a human), more natural /snap/core/1234<subdirectory>.

This is also discussed in the pull request to snapd, where it was originally discovered:
https://github.com/snapcore/snapd/pull/3621#discussion_r137389940