whites out /lib/modules and /usr/src

[Evan]

In trying to snap bcc (https://github.com/iovisor/bcc), I ran into the following:

chdir(/lib/modules/4.4.0-15-generic/build): No such file or directory
Traceback (most recent call last):
  File "/snap/bcc/current/usr/share/bcc/tools/biotop", line 152, in <module>
    """, debug=0)
  File "/snap/bcc/current/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 126, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module

Digging further, I discovered that ubuntu-core-launcher bind-mounts /lib and /usr in the snap's namespace. This works fine for most things, but bcc is expecting to find the kernel headers symlinked from /lib/modules/*/build to /usr/src. Both of these directories are rightfully empty in the OS snap.

Given that this code path is just used for Ubuntu Classic, we could instead bind mount one level down, skipping over /usr/src and /lib/modules (as those will match the running kernel).

[Mark Shuttleworth (sabdfl)]

Good catch, thanks for tackling bcc Ev, that's the stuff of movies :)

[Gustavo Niemeyer (niemeyer)]

We cannot bind-mount one level down because that would expose content we don't want snaps to depend upon. That said, for /lib/modules we can probably just bind mount it always as devices will have equivalent content in the same place via the kernel module (we need to verify if that's currently exposed to snaps, though).

For kernel sources it may be slightly more complex since we don't want to ship kernel sources at all times, nor to have snaps depending on content which might be there or not. We might expose that via an interface that gives access to that content. In the sprint we quickly discussed the idea of adding another security backend specialized in arbitrary bind-mounts, and Zygmunt has already started some research on that. So this might turn out to be very easy to support once that feature lands.

[Gustavo Niemeyer (niemeyer)]

Sorry, on the first sentence s/kernel module/kernel snap/.

[Zygmunt Krynicki (zyga)]

I agree with Gustavo. I think there are two bugs here:

 - kernel sources could be an interface exposed on the classic systems
 - /lib/modules could be something we bind mount

I'd like to re-purpose this bug for the 2nd task and ask the reporter to report the kernel sources interface separately.

[Zygmunt Krynicki (zyga)]

So currently all of /lib is now allowed for reading (this is a change on snapd, not snap-confine though). I'm going to close this bug now.

[Zygmunt Krynicki (zyga)]

I'd like to clarify my earlier comment. This is the state of the code as of right now:

snapd allows snaps to read /lib but the content there is always that of the core snap (it is always the same on both classic and all-snap systems). The /usr/src interface doesn't exist yet, it could be an interesting use of the content-sharing actually. The same goes for accessing /lib/modules.

Note that /lib/modules should perhaps be handled by snap-confine so that the behavior is consistent across systems.

I took the liberty to fork this issue into two new bugs where we can discuss and track this:

/lib/modules issue: https://bugs.launchpad.net/snap-confine/+bug/1597839
/usr/src issue: https://bugs.launchpad.net/snappy/+bug/1597842