autogenerated per-snap snap-update-ns apparmor profile may contain many duplicate mount rules causing excessive parser memory usage
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
AppArmor |
New
|
Undecided
|
Unassigned | ||
snapd |
Fix Released
|
Medium
|
Zygmunt Krynicki |
Bug Description
I decided to profile the memory usage of apparmor_parser for various snap profiles and discovered that some snaps' auto-generated snap-update-ns profile had thousands of duplicate rules (particularly for snaps using the content interface multiple times). This caused a slowdown of the parser to parse the file as well as excessive memory usage. Eg:
$ for i in /var/lib/
= /var/lib/
wall: 0:00.43, mem: 22096 Kb
= /var/lib/
wall: 0:04.88, mem: 1541680 Kb
Whoa, almost 5 seconds to compile on a modern i7 amd64 system and 1.5G of memory used.
$ wc -l /var/lib/
17547 /var/lib/
That's a big profile. :)
Using sort -u on the profile, and then putting the preamble (ie "#include <tunables/global>" and "profile ... {") and the trailing "}" in place so that it is a proper profile, I see:
$ wc -l ./snap-
662 ./snap-
$ /usr/bin/time -f "wall: %E, mem: %M Kb" apparmor_parser -QTK -Ono-expr-simplify ./snap-
wall: 0:00.39, mem: 36276 Kb
That's a *lot* better. Note, while my machine has 4 CPUs, using -j1 doesn't make a difference:
$ /usr/bin/time -f "wall: %E, mem: %M Kb" apparmor_parser -j1 -QTK -Ono-expr-simplify /var/lib/
wall: 0:05.32, mem: 1541236 Kb
Creating two tasks since there are 2 bugs here: snapd should not be creating a profile with so many duplicate rules and apparmor's deduplication would ideally not use so much memory.
For snapd, I suggest that when auto-generating the snap-update-ns profile, it (efficiently) dedupes before splatting out to the disk. This will cause speedups and lower memory consumption across install and refresh regardless of parser version (not to mention, smaller file sizes).
Attached are the two profiles above generated with snapd 2.42:
$ snap version
snap 2.42
snapd 2.42
series 16
ubuntu 19.04
kernel 5.0.0-29-generic
description: | updated |
description: | updated |
description: | updated |
summary: |
autogenerated per-snap snap-update-ns apparmor profile may contain many - duplicate rules causing excessive parser memory usage + duplicate mount rules causing excessive parser memory usage |
description: | updated |
Changed in snapd: | |
status: | New → In Progress |
assignee: | nobody → Zygmunt Krynicki (zyga) |
Changed in snapd: | |
milestone: | none → 2.43 |
Changed in snapd: | |
status: | In Progress → Fix Committed |
The apparmor_parser does a dedup pass and a simplification pass before state machine build and minimization.
Unfortunately the parser is not applying dedup to the mount rules, and the simplification pass is disabled.
-O expr-simplify: 2.45s RSS: 151.3 MB
-O no-expr-simplify 3.24s RSS: 1.2 GB
The reported difference between sorted/uniqued can be fixed by fixing the dedup pass for mount rules.