installing snap with layout on /etc/ld.so.cache results in deleted mount

[Ian Johnson]

After installing a snap that modifies the file $SNAP_DATA/etc/ld.so.cache which is binded onto /etc/ld.so.cache using layouts, the mount setup for layouts at /etc/ld.so.cache shows up as //deleted in /proc/self/mountinfo for the snap's processes. See:

```
$ snap install update-ldconfig_0.1_amd64.snap --dangerous
$ snap connect update-ldconfig:mount-observe
$ sudo snap run --shell update-ldconfig -c "cat /proc/self/mountinfo | grep deleted"
$ sudo update-ldconfig
+ '[' '!' -f /var/snap/update-ldconfig/x2/etc/ld.so.cache ']'
+ mkdir -p /var/snap/update-ldconfig/x2/etc/ld.so.conf.d
+ cp -r /etc/ld.so.conf.d/fakeroot-x86_64-linux-gnu.conf /etc/ld.so.conf.d/i386-linux-gnu.conf /etc/ld.so.conf.d/libc.conf /etc/ld.so.conf.d/x86_64-linux-gnu.conf /etc/ld.so.conf.d/zz_i386-biarch-compat.conf /etc/ld.so.conf.d/zz_x32-biarch-compat.conf /var/snap/update-ldconfig/x2/etc/ld.so.conf.d
+ cp -r /etc/ld.so.conf /var/snap/update-ldconfig/x2/etc/ld.so.conf
++ echo /var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void::/snap/update-ldconfig/x2/lib:/snap/update-ldconfig/x2/usr/lib:/snap/update-ldconfig/x2/lib/x86_64-linux-gnu:/snap/update-ldconfig/x2/usr/lib/x86_64-linux-gnu
++ sed -e s/:://
+ LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void/snap/update-ldconfig/x2/lib:/snap/update-ldconfig/x2/usr/lib:/snap/update-ldconfig/x2/lib/x86_64-linux-gnu:/snap/update-ldconfig/x2/usr/lib/x86_64-linux-gnu
+ IFS=:
+ read -ra PATHS
+ ldconfig -C /var/snap/update-ldconfig/x2/etc/ld.so.cache -c /var/snap/update-ldconfig/x2/etc/ld.so.conf /var/lib/snapd/lib/gl /var/lib/snapd/lib/gl32 /var/lib/snapd/void/snap/update-ldconfig/x2/lib /snap/update-ldconfig/x2/usr/lib /snap/update-ldconfig/x2/lib/x86_64-linux-gnu /snap/update-ldconfig/x2/usr/lib/x86_64-linux-gnu
$ sudo snap run --shell update-ldconfig -c "cat /proc/self/mountinfo | grep deleted"
4696 4851 259:6 /var/snap/update-ldconfig/x2/etc/ld.so.cache//deleted /etc/ld.so.cache rw,relatime master:1 - ext4 /dev/nvme1n1p1 rw,errors=remount-ro
```

This is the snap.yaml:

```
name: update-ldconfig
version: '0.1'
summary: update-ldconfig
description: update-ldconfig
apps:
  update-ldconfig:
    command: update-ldconfig.sh
    command-chain:
    - snap/command-chain/snapcraft-runner
    plugs:
    - mount-observe
architectures:
- amd64
base: core18
confinement: strict
grade: devel
layout:
  /etc/ld.so.cache:
    bind-file: $SNAP_DATA/etc/ld.so.cache
```

with update-ldconfig.sh defined as :

```
#!/bin/bash -ex

# make sure our fake ld.so.cache exists
if [ ! -f "$SNAP_DATA/etc/ld.so.cache" ]; then
    echo "snap private ld.so.cache doesn't exist!"
    exit 1
fi

# copying the conf dir from real /etc into $SNAP_DATA
mkdir -p "$SNAP_DATA/etc/ld.so.conf.d"
cp -r /etc/ld.so.conf.d/* "$SNAP_DATA/etc/ld.so.conf.d"
cp -r /etc/ld.so.conf "$SNAP_DATA/etc/ld.so.conf"

# delete empty entries in the LD_LIBRARY_PATH
# i.e. change "/a/b/c:/1/2/3::/other" into "/a/b/c:/1/2/3:/other"
LD_LIBRARY_PATH=$(echo "$LD_LIBRARY_PATH" | sed -e "s/:://")

# run ldconfig on our LD_LIBRARY_PATH lib dirs
IFS=':' read -ra PATHS <<< "$LD_LIBRARY_PATH"
ldconfig -C "$SNAP_DATA/etc/ld.so.cache" -c "$SNAP_DATA/etc/ld.so.conf" "${PATHS[@]}"

```

I tried this both on stable and with master build of snapd. I can't seem to reproduce this with other paths so if I had to guess this has something to do with /etc/ld.so.conf already existing.

Zygmunt any idea if this fixed by the MS_SHARED patch you have (or one of the other PR's you have in flight for mount namespace bugs)?

[Zygmunt Krynicki (zyga)]

This is not going to be fixed by MS_SHARED

I think the following happens:

touch source target
mount --bind source target
rm source
# now target is a deleted mount

This is what ldconfig does, it seems to rewrite the file by unlinking the original at some point. I didn't test this theory but it is the mechanism in which deleted mounts can be created.

As a workaround I would suggest to use a symlink: /etc/ld.so.cache -> $SNAP_DATA/ld.so.cache and then run ldconfig on the $SNAP_DATA location.

[Maciej Borzecki (maciek-borzecki)]

Could we warn about such known cases when packing the snap?

[Zygmunt Krynicki (zyga)]

FYI I'm not sure if we can fix this.

[Ian Johnson (anonymouse67)]

Some additional context / workarounds available here: https://forum.snapcraft.io/t/layout-bind-file-does-not-reflect-changes-to-source-file/17524