Stopping container signal blocked by AppArmor on Ubuntu

[Gunnar]

Issue originally reported here: https://github.com/moby/moby/issues/47720

### Description

When a container is stopped, the quit and kill signal get blocked by AppArmor.

Workaround:
The only way I have found to bypass this issue is to disable apparmor for docker by setting environment variable `container` to any value.

See this: https://github.com/moby/moby/issues/33060#issuecomment-2056845779 for details.

### Reproduce

Setup clean version of Ubuntu 23.10

`apt update`
`apt full-upgrade -y`
`apt install -y docker.io`
`reboot`

Try this multiple times:
`time docker stop $(docker run --rm -d nginx)`

Observe time taken is around 12 seconds.

Now do my workaround linked above, or disable apparmor system wide.

Now try the same again a few times:
`time docker stop $(docker run --rm -d nginx)`

Observe time taken is only around 1 second.

My workaround is required because [it is not possible to modify, edit, view or anything the docker-default apparmor profile https://github.com/moby/moby/issues/33060, or you can disable AppArmor system wide.

### Expected behavior

`docker stop` should not be blocked by AppArmor.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Seth Arnold (seth-arnold)]

AppArmor's signal handling is a bit more involved than eg capabilities or file accesses: both the sender profile and receiver profile need to have signal rules to allow sending the signal or receiving the signal, as appropriate.

23.10 and 24.04 LTS have introduced restrictions on unprivileged namespaces to try to mitigate against kernel exploits. The details have changed between 23.10 and 24.04 LTS, so it's possible that upgrading to 24.04 LTS will be sufficient to fix this -- especially if the AppArmor profiles have been updated during the development cycle.

I'm not sure what exactly to suggest as I don't know the various kinds of Docker available, where the profiles live, etc. But hopefully these hints will help you get to a fix.

Thanks

[Tomáš Virtus (virtustom)]

There's a fix proposed to upstream: https://github.com/moby/moby/pull/47749

The commit message describes the cause.

These bugs have the same cause:
- https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2039294
- https://bugs.launchpad.net/ubuntu/+source/libpod/+bug/2040483

The latter doesn't have a fix submitted yet. I'm working on that.

[Tomáš Virtus (virtustom)]

I'll copy the workaround I mentioned in #2039294 here:

As a temporary workaround, put the file I have attached to /etc/apparmor.d/docker-default and load it with "apparmor_parser -Kr /etc/apparmor.d/docker-default". It will make dockerd skip loading its builtin profile as docker-default. It will also stick across reboots. The only difference between the builtin profile and the attached one are the following rules:

  # runc may send signals to container processes
  signal (receive) peer=runc,

Add similar line for crun if you're using crun.

[Lucas Kanashiro (lucaskanashiro)]

Thanks for providing the workaround Tomáš! I can confirm that it works in Noble, but for me, even using the profile you provided in comment #4, the command below takes more or less 10 seconds (against 12 seconds when the containers are killed with SIGKILL):

root@docker-apparmor:~# time docker stop $(docker run --rm -d nginx)
81948b000a5510fe4d59c476d783c7f4d8946f86ffa916d4b317678c74b54ffb

real 0m12.538s
user 0m0.015s
sys 0m0.027s
[ ... download profile from comment #4 ... ]
root@docker-apparmor:~# apparmor_parser -Kr /etc/apparmor.d/docker-default
root@docker-apparmor:~# time docker stop $(docker run --rm -d nginx)
ef2b4c117fa48954288c418d80088ce88fc5233fc13a7084003cb5c980e42369

real 0m10.563s
user 0m0.016s
sys 0m0.027s
root@docker-apparmor:~# time docker stop $(docker run --rm -d nginx)
a8ab8186872d856368ad4b627fa36fa5b0fff78ac5f6ddc9db3e0545f744b741

real 0m10.514s
user 0m0.011s
sys 0m0.031s
root@docker-apparmor:~# time docker stop $(docker run --rm -d nginx)
42d8afedec05f31ba1999ffbad2e7696eaf295f150e3b5f26a8e5e3cd27a2406

real 0m10.503s
user 0m0.019s
sys 0m0.024s

[Tomáš Virtus (virtustom)]

@lucaskanashiro,

I think you are trying top stop the container too soon after it's created. The container receives SIGTERM from docker before is sets up signal handlers, and because it's PID 1, the signal is ignored. Runc then kills it with SIGKILL after 10s.

Try with sleep:

root@cloudimg:~# time docker stop $(docker run --rm -d nginx)
d975cb1c9c088db8ee5ce6d79c36b2931675cfb3ab244eb5effc8f605b357e5f

real 0m10.875s
user 0m0.059s
sys 0m0.066s
root@cloudimg:~# time docker stop $(docker run --rm -d nginx && sleep 1)
bffbc7672f45fcda107371f3d95779e3a45152d03e7a220532ed94dcda4b72a6

real 0m1.750s
user 0m0.066s
sys 0m0.076s