shiftfs breaks paths in /proc/*/maps

[Simon Fels]

# Overview

When enabling shiftfs for LXD we get incorrect paths in /proc/*/maps within a container:

root@anbox0:~# cat /proc/self/maps
55930e600000-55930e608000 r-xp 00000000 00:51 14509 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/bin/cat
55930e807000-55930e808000 r--p 00007000 00:51 14509 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/bin/cat
55930e808000-55930e809000 rw-p 00008000 00:51 14509 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/bin/cat
55930fd3c000-55930fd5d000 rw-p 00000000 00:00 0 [heap]
7fe736be2000-7fe736dc9000 r-xp 00000000 00:51 14226 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/libc-2.27.so
7fe736dc9000-7fe736fc9000 ---p 001e7000 00:51 14226 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/libc-2.27.so
7fe736fc9000-7fe736fcd000 r--p 001e7000 00:51 14226 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/libc-2.27.so
7fe736fcd000-7fe736fcf000 rw-p 001eb000 00:51 14226 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/libc-2.27.so
7fe736fcf000-7fe736fd3000 rw-p 00000000 00:00 0
7fe736fd3000-7fe736ffc000 r-xp 00000000 00:51 14139 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/ld-2.27.so
7fe7371ba000-7fe7371de000 rw-p 00000000 00:00 0
7fe7371fc000-7fe7371fd000 r--p 00029000 00:51 14139 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/ld-2.27.so
7fe7371fd000-7fe7371fe000 rw-p 0002a000 00:51 14139 /var/snap/lxd/common/lxd/storage-pools/default/containers/anbox0/rootfs/lib/x86_64-linux-gnu/ld-2.27.so
7fe7371fe000-7fe7371ff000 rw-p 00000000 00:00 0
7ffcd50a0000-7ffcd50c1000 rw-p 00000000 00:00 0 [stack]
7ffcd51d2000-7ffcd51d6000 r--p 00000000 00:00 0 [vvar]
7ffcd51d6000-7ffcd51d8000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]

When shiftfs is disabled the paths look all correct and are relative to the root of the container

root@anbox0:~# cat /proc/self/maps
557aaa200000-557aaa208000 r-xp 00000000 00:48 14509 /bin/cat
557aaa407000-557aaa408000 r--p 00007000 00:48 14509 /bin/cat
557aaa408000-557aaa409000 rw-p 00008000 00:48 14509 /bin/cat
557aabaa5000-557aabac6000 rw-p 00000000 00:00 0 [heap]
7f4820526000-7f482070d000 r-xp 00000000 00:48 14226 /lib/x86_64-linux-gnu/libc-2.27.so
7f482070d000-7f482090d000 ---p 001e7000 00:48 14226 /lib/x86_64-linux-gnu/libc-2.27.so
7f482090d000-7f4820911000 r--p 001e7000 00:48 14226 /lib/x86_64-linux-gnu/libc-2.27.so
7f4820911000-7f4820913000 rw-p 001eb000 00:48 14226 /lib/x86_64-linux-gnu/libc-2.27.so
7f4820913000-7f4820917000 rw-p 00000000 00:00 0
7f4820917000-7f4820940000 r-xp 00000000 00:48 14139 /lib/x86_64-linux-gnu/ld-2.27.so
7f4820afe000-7f4820b22000 rw-p 00000000 00:00 0
7f4820b40000-7f4820b41000 r--p 00029000 00:48 14139 /lib/x86_64-linux-gnu/ld-2.27.so
7f4820b41000-7f4820b42000 rw-p 0002a000 00:48 14139 /lib/x86_64-linux-gnu/ld-2.27.so
7f4820b42000-7f4820b43000 rw-p 00000000 00:00 0
7ffd12ead000-7ffd12ece000 rw-p 00000000 00:00 0 [stack]
7ffd12fd8000-7ffd12fdc000 r--p 00000000 00:00 0 [vvar]
7ffd12fdc000-7ffd12fde000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]

To reproduce:

1. Use Ubuntu 20.04 or newer inluding shiftfs
2. Install LXD (4.0.x is good enough) and enable shiftfs support

$ snap install lxd
$ snap set lxd shiftfs.enable=true
$ snap restart lxd
$ lxd init --auto

3. Start a container and check /proc/self/maps
$ lxc launch ubuntu:f test0
$ lxc exec test0 -- cat /proc/self/maps

4. Turn shiftfs back off

$ snap set lxd shiftfs.enable=false
$ snap restart lxd

5. Check /proc/self/maps again inside the container

$ lxc exec test0 -- cat /proc/self/maps

This can be reproduced with both the 5.4 and 5.13 kernels on Ubuntu 20.04 and also with 5.15 on a fresh jammy installation with a ZFS storage pool

# Impact

We see customer workloads crashing because of incorrect paths in /proc/*/maps as applications are using the paths they read from procfs to access things

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1968027

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Simon Fels (morphis)]

Attaching logs fails when `apport-collect 1968027` wants to authorized with my LP with an "Not allowed error".