Issues when activating a large number of applications at the same time

[Dave Morley]

STORY:
Wanted to see if there were a limit to the number of apps you could install and run at once.

STEPS:
1. Flash latest core
2. snap download hello-world
3. sudo unsquashfs hello-world_27.snap
4. sudo chown -R <user:user> sqaushfs-root/
5. (set -e; for i in {1..258}; do sed -i -e "s/^name: hello-world.*$/name: hello-world-$i/" squashfs-root/meta/snap.yaml; mksquashfs squashfs-root/ hello-world-$i.snap -noappend -comp xz -all-root -no-xattrs; done )
6. for i in `find . -name \*.snap`;do sudo snap install --dangerous $i; done
7. for i in {1..258}; do hello-world-$i.sh & done

EXPECTED:
I expect all that snaps to be runable

ACTUAL:
Part way through the mounts nolonger mount and run exiting the mount.

EXAMPLE_ERROR:
Launching a shell inside the default app confinement. Navigate to your
app-specific directories with:

  $ cd $SNAP
  $ cd $SNAP_DATA
  $ cd $SNAP_USER_DATA

cannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directorycannot acquire exclusive lock for namespace group directory. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
. errmsg: Interrupted system call
Launching a shell inside the default app confinement. Navigate to your

[Zygmunt Krynicki (zyga)]

It seems this bug affects core systems only (this is just a theory) and the solution is to make a few more things unbindable. I'm investigating the cause right now.

[Zygmunt Krynicki (zyga)]

This is a pastebin from a system with a few snaps and three active mount namespaces: http://paste.ubuntu.com/23353845/

[Seth Arnold (seth-arnold)]

Once upon a time the Linux kernel defaulted to only 16 loopback mounts at a time; is that still the case? Or is it a coincidence that your paste shows 16 loopbacks in use?

Thanks

[John Lenton (chipaca)]

Coincidence; there were over 200 loopbacks active in the system.

[Zygmunt Krynicki (zyga)]

This was traced to running out of memory and addressed via kernel bug https://bugs.launchpad.net/snappy/+bug/1636847

There may be an issue left to investigate that affects core systems and /writable. I will add a spread test and either close this issue (via duplicate link) or work on a remedy for /writable.

[Zygmunt Krynicki (zyga)]

I've closed this bug. The root cause was a kernel configuration option that used excessive memory for each mounted squashfs file. This has been fixed long ago now.