Apparmor breaking nsjail in AOSP

[Alexander Koskovich]

Build sandboxing in AOSP is broken after updating to 24.04 with the following denials:

[ 182.439078] audit: type=1400 audit(1714265880.641:449): apparmor="AUDIT" operation="userns_create" class="namespace" info="Userns create - transitioning profile" profile="unconfined" pid=8514 comm="nsjail" requested="userns_create" target="unprivileged_userns"
[ 182.439945] audit: type=1400 audit(1714265880.642:450): apparmor="DENIED" operation="capable" class="cap" profile="unprivileged_userns" pid=8515 comm="nsjail" capability=6 capname="setgid"
[ 182.439972] audit: type=1400 audit(1714265880.642:451): apparmor="DENIED" operation="mount" class="mount" info="failed mntpnt match" error=-13 profile="unprivileged_userns" name="/" pid=8515 comm="nsjail" flags="rw, rprivate"

This seems to come from the following change earlier this year: https://gitlab.com/apparmor/apparmor/-/commit/789cda2f089b3cd3c8c4ca387f023a36f7f1738a

[John Johansen (jjohansen)]

Commit 789cda2f089b3cd3c8c4ca387f023a36f7f1738a only controls the behavior of unprivileged user namespace mediation.

With the unprivileged_userns profile loaded, when a user namespace is created by an unprivileged unconfined application the task will be transitioned into the unprivileged_userns profile. The unprivileged_userns profile will then deny privileged operations capability, mount etc.

Without the unprivileged_userns profile loaded, the creation of the user namespace will be denied.

Through experimentation we have learned that many applications behave better (handle the errors better, eg. qtwebkit will handle the error and fallback to using a sandbox without usernamespaces while without the profile it crashes) with the unprivileged_userns loaded. So that has become the default behavior.

You can experiment with changing the behavior by manually unloading the unprivileged_userns profile using

  sudo apparmor_parser -R /etc/apparmor.d/unprivileged_userns

nsjail will likely require a profile to work, please see https://discourse.ubuntu.com/t/noble-numbat-release-notes/39890#unprivileged-user-namespace-restrictions-15

[Alexander Koskovich (nexusprism)]

Thanks, I took a look at creating a profile for nsjail, but I'm a bit confused on how to associate it with the app?

Because nsjail is a prebuilt in AOSP's source code that means it could be litteraly anywhere on the user's system, e.g:
~/android-14.0.0_r1/prebuilts/build-tools/linux-x86/bin/nsjail
~/android-13.0.0_r1/prebuilts/build-tools/linux-x86/bin/nsjail
~/android-12.0.0_r1/prebuilts/build-tools/linux-x86/bin/nsjail

```
profile nsjail /**/prebuilts/build-tools/linux-x86/bin/nsjail flags=(unconfined) {
```

I tested the above and it works, but is there a better way to do this? Feels dirty and not what apparmor people would want.

[John Johansen (jjohansen)]

running privileged applications out of home is dirty. But it is the situation we are in with user namespaces and app images as well. Ubuntu will not ship a profile for a privileged executable in the users home or a writable location of an unprivileged user. As this can be leveraged to by-pass the restriction, or it requires us to expand user mediation in such a way that user writable locations with profiles defined become privileged. Atm we are not adding addition restriction to the user. This allows the user to define a profile that allows by-passing the restriction. A user opting to create a profile in a user writable location is less dangerous as the location becomes non-standard so it becomes harder to exploit. It also requires the user to take a deliberate privileged action to add the profile.

Generally for the nsjail profile an attachment like

  @{HOME}/android-*/prebuilts/build-tools/linux-x86/bin/nsjail

is slightly better, but still not great. Atm it is very close to the same, but there are improvements coming that will tighten @{HOME} to a user specific kernel variable which will be better than /**.

The other way to handle this would be setting the security xattr and using that as part of the attachment.

```
  sudo setfattr -n security.apparmor -v nsjail
```

and define the profile as something like (you can make the path more specific if you want).

```
  profile nsjail /**/nsjail xattrs=(security.apparmor="nsjail") flags=(unconfined) {
```

[Alexander Koskovich (nexusprism)]

To clarify, this is not something that can be solved upstream in apparmor, and a profile can't be accepted due to the nature of the path location?

I'm really trying to avoid a situation where we need to add additional instructions after syncing AOSP just for Ubuntu users.

One idea for this was to take nsjail and package it in Debian and remove it from AOSP prebuilts, that way a proper profile could be upstreamed since the path would be static, but that wouldn't address all previous versions of Android.

Google has also been trying to move all binaries required for AOSP compile into the tree itself for more reproducible builds, so I'm not even sure if they'd accept that.

[John Johansen (jjohansen)]

> To clarify, this is not something that can be solved upstream in apparmor, and a profile can't be accepted due to the nature of the path location?

correct, if it is a unprivileged user writable location it can't be fixed entirely upstream. It is possible for us to ship a profile that is disabled in some way but that takes a privileged user action to enable. Eg. we could ship a profile using the xattrs attachment from above, then the user would be responsible for setting the xattr with setfattr.

packaging nsjail is an option for Ubuntu but like you said it wouldn't directly address previous versions and AOSP probably wouldn't like it. With that said this isn't going to be an Ubuntu only restriction, the security community in general is looking at different ways of restricting unprivileged user namespaces. SElinux has picked up some ability to mediate them, but isn't really applying it in policy yet. The OSS email list (<email address hidden>) has been discussing other options as well. The number of exploit chains associated with them has forced us to start locking them down. The AppArmor solution will be available to other distros as well, it already available upstream in the kernel and apparmor 4.0.

AppArmor side there is work on aa-notify that we are looking at SRUing. That will help desktop users if they have it installed. Where they can get a notification that will take them to a simple gui that will allow them to click enable (with a password) instead of having to know the details underneath. It won't be integrated into the security center or pretty. But a little better than the current situation for the user.