core18 base on core 16 missing firmware

[Ondrej Kubik]

snap version:
snap 2.38~pre1+git1197.b5e5471~ubuntu16.04.1
snapd 2.38~pre1+git1197.b5e5471~ubuntu16.04.1
series 16
kernel 4.4.0-143-generic

Precondition:
 - Ubuntu Core 16 system
 - snap using base core 18
 - snap trying to load firmware file, e.g. wlan firmware

Expected behaviour:
 - works

Actual behaviour:
 - firmware file is missing

Actually entire firmware directory from kernel snap is not mapped into core 18 base when on core 16 system

I assume this be generalised as:
 When using base which has not been used as boot rootfs, kernel snap overlay is not applied. e.g, /lib/firmware, /lib/modules are empty

[Ondrej Kubik (ondrak)]

Simple test:
Core 16 system:

snap-A: base core16
snap-B: base core18

$ snap run --shell snap-A.app
ls /lib/firmware
  - expected populated directory

$ snap run --shell snap-B.app
ls /lib/firmware
  - unexpected, empty directory

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

I see the same issue in the reverse case, Ubuntu Core 18 system and a snap with a core16 base. Even more, it looks like /lib/firmware is also bad for snaps with core18 base in UC18.

[Ondrej Kubik (ondrak)]

you first case (core 18 with core 16 base) is aligned with generic
statement at the end of the description

Though core 18 + core18 is surprising one. And I can confirm this is the
case. Just run test on core 18 + core 18 base and indeed /lib/firmware is
empty

On Wed, Mar 20, 2019 at 4:05 PM Alfonso Sanchez-Beato <
<email address hidden>> wrote:

> I see the same issue in the reverse case, Ubuntu Core 18 system and a
> snap with a core16 base. Even more, it looks like /lib/firmware is also
> bad for snaps with core18 base in UC18.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1821023
>
> Title:
> core18 base on core 16 missing firmware
>
> Status in Snappy:
> Triaged
>
> Bug description:
> snap version:
> snap 2.38~pre1+git1197.b5e5471~ubuntu16.04.1
> snapd 2.38~pre1+git1197.b5e5471~ubuntu16.04.1
> series 16
> kernel 4.4.0-143-generic
>
> Precondition:
> - Ubuntu Core 16 system
> - snap using base core 18
> - snap trying to load firmware file, e.g. wlan firmware
>
> Expected behaviour:
> - works
>
> Actual behaviour:
> - firmware file is missing
>
> Actually entire firmware directory from kernel snap is not mapped into
> core 18 base when on core 16 system
>
> I assume this be generalised as:
> When using base which has not been used as boot rootfs, kernel snap
> overlay is not applied. e.g, /lib/firmware, /lib/modules are empty
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/snappy/+bug/1821023/+subscriptions
>

[Jamie Strandboge (jdstrand)]

Please note that even if the files are available in the snap's runtime at a predictable location, we will not typically want snaps to be able to load firmware themselves for all the reasons that we don't want them to load kernel modules themselves. We can consider adding a firmware-control interface that allows it as an escape hatch, but I'd rather see either the gadget snap add support for this and/or a firmware backend is created that functions like the kmod one, where we can have snapd load firmware on behalf of the snap. Typical snaps shouldn't be expected to load firmware to function. Something else might be needed based on requirements.

[Ondrej Kubik (ondrak)]

correct, I'd still expect confinement limit what can be actually done.
In use case I have found, snap is using ifconfig, and has all needed network control plugs.
It was simple call "ifconfig wlan1 0.0.0.0" which was failing. I'm not sure if ifconfig was loading firmware as I assume it should have been loaded by then, or if it parses firmware file for some information, either way it was failing as it was unable to locate the firmware.
Once firmware file was there there was no confinement denial, so I assume it was not doing anything unusual.

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

The problem I have seen is that the first usage from userspace after boot of some HW triggers loading the firmware for that device. It is not the snap explicitly asking the kernel to load FW. The issue is that the kernel tries to get the firmware files looking at the filesystem layout seen by the process calling the kernel.

I have seen this bug in two apparently innocuous cases, from confined processes:

* Trying to connect to a wifi AP
* When using CUDA

If I try any of these things from a non-confined process, things are fine, if then I do the same things from the confined processes, things are fine too. The bug appears only in the first call when the drivers decide to load the FW.

The ideal way to resolve would be to prevent the kernel from using the confined process filesystem layout, but I am not sure if that is possible.

[Ian Johnson (anonymouse67)]

Is this slated to be fixed in 2.39.2 or has this now slipped to 2.40?

[Zygmunt Krynicki (zyga)]

Hello

I had a look at this issue just now:

- On core16 systems /lib/firmware is mounted from /firmware of the kernel snap
- On core18 systems /lib/firmware is mounted from /firmware of the kernel snap

So far so good

Inside the per-snap mount namespace, the situations is slightly different:

- On core16 systems with an app snap using core16/core as bases /lib/firmware is "natively" taken from the initial mount namespace, that is, it is the /firmware directory of the kernel snap
- On core16 systems with an app snap using core18 as base /lib/firmware is not mounted
- On core18 systems with an app snap using any type of base /lib/firmware is not mounted

As a solution we should add lib/firmware to bind mount set in mount-support.c in normal mode.

[Zygmunt Krynicki (zyga)]

I sent a proposal to fix this https://github.com/snapcore/snapd/pull/7104