specifying -O no-expr-simplify results in cache miss

[Jamie Strandboge]

[Impact]

 * AppArmor 2.13 unconditionally invalidates its cache when parser options are specified. To decrease compile times for ARM systems, -O no-expr-simplify has been used in Ubuntu for click and snap policy for many years, but was temporarily disabled during the disco development release with the 2.13 upload so caching properly worked everywhere. Now that a simple upstream workaround is available (and already in eoan), we'd like to apply the upstream patch and re-enable -O no-expr-simplify.
 * A condition of the AppArmor 2.13 feature freeze exception was to fix this bug in SRU to re-enable -O no-expr-simplify. This will help compile times for all architectures with default and typical AppArmor policy (ie, a mixture of (distro) system and snap policy), but especially ARM systems with snaps where the improvement could be in terms of minutes saved.
 * Specifically, the upstream patch workaround to no longer unconditionally skip reading the cache when parser options are specified. It also re-enables an existing quilt patch to update /etc/apparmor/parser.conf to use no-expr-simplify.

[Test Case]

# setup
$ mkdir -p /tmp/aa/cache /tmp/aa/profiles
$ cp /etc/apparmor.d/sbin.dhclient /tmp/aa/profiles/

# no options, no cache, expect a miss and to write
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

# no options, cache, expect a hit
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache hit: /tmp/aa/cache/26b63962.0/sbin.dhclient

# reset
$ rm -rf /tmp/aa/cache/*

# options, no cache, expect a miss and to write
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load -O no-expr-simplify --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

# options, cache, expect a hit
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load -O no-expr-simplify --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient # SHOULD BE A HIT
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

Same thing happens if omitting -O no-expr-simplify but add Optimize=no-expr-simplify to /etc/apparmor/parser.conf.

[Regression Potential]

The regression potential is considered low since the patch is simple and easily verifiable. If something went wrong, it would be around cache invalidation which the above test case will demonstrate it works correctly.

# Original description

With 2.13.2 and the most recent testsuite patches from the 2.13 branch, I find that the cache works correctly when no options are specified. Eg

# setup
$ mkdir -p /tmp/aa/cache /tmp/aa/profiles
$ cp /etc/apparmor.d/sbin.dhclient /tmp/aa/profiles/

# no options, no cache, expect a miss and to write
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

# no options, cache, expect a hit
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache hit: /tmp/aa/cache/26b63962.0/sbin.dhclient

# reset
$ rm -rf /tmp/aa/cache/*

# options, no cache, expect a miss and to write
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load -O no-expr-simplify --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

# options, cache, expect a hit
$ /sbin/apparmor_parser -k --write-cache --cache-loc=/tmp/aa/cache --skip-kernel-load -O no-expr-simplify --add -- /tmp/aa/profiles
Cache: added primary location '/tmp/aa/cache'
Cache miss: /tmp/aa/profiles/sbin.dhclient # SHOULD BE A HIT
Wrote cache: /tmp/aa/cache/26b63962.0/sbin.dhclient

Same thing happens if omitting -O no-expr-simplify but add Optimize=no-expr-simplify to /etc/apparmor/parser.conf.

[Jamie Strandboge (jdstrand)]

I verified this does not affect Ubuntu 18.04 which uses 2.12.

[Jamie Strandboge (jdstrand)]

I can say that this was also not introduced by these latest commits:

5704fba8d935e8ea31c769e88f4a5ddfa9696a99 library: fix segfault in overlaydirat_for_each
9374f419a0169e1e739a1aac567967841343d85d tests: Add option to dump policy cache dir with the libapparmor wrapper
68eb3be2ae8777527159ce63ea399b3e108475ae tests: Teach aa_policy_cache.sh about the cache location and subdirs
01aec04bd6457441771cda028418564606c3a653 libapparmor: Fix segfault when loading policy cache files
b502110dcf629a79960ec90e15d1d50772cd91d8 libapparmor: Fix variable name overlap in merge() macro

[Jamie Strandboge (jdstrand)]

Adding an Ubuntu 19.04 task in anticipation of the 2.13.2 upload.

[Jamie Strandboge (jdstrand)]

Upstream patch is here: https://gitlab.com/apparmor/apparmor/merge_requests/385

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.13.2-9ubuntu7

---------------
apparmor (2.13.2-9ubuntu7) eoan; urgency=medium

  * lp1820068.patch: don't skip read cache when options are set (LP: #1820068)
  * reenable ubuntu/parser-conf-no-expr-simplify.patch

 -- Jamie Strandboge <email address hidden> Thu, 06 Jun 2019 21:46:34 +0000

[Łukasz Zemczak (sil2100)]

Hello Jamie, or anyone else affected,

Accepted apparmor into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/2.13.2-9ubuntu6.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Jamie Strandboge (jdstrand)]

FYI, I tested this and 2.13.2-9ubuntu6.1 fixes this bug.

I also executed https://wiki.ubuntu.com/Process/Merges/TestPlans/AppArmor (sans dbus optional bits) and everything passed.

Lastly, I wanted to double check the performance impact of no-expr-simplify on amd64 especially as it pertains to cloud and found it was slightly faster for default cloud, lxd and server policy.

(It was slower on desktop, as expected and in the expected way due to the somewhat pathological evince profile. This has been the historical acceptable tradeoff to help ARM devices (note that because no-expr-simplify is faster with snap policy, with a handful of additional snaps, desktop breaks even then eventually gets faster too).

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.13.2-9ubuntu6.1

---------------
apparmor (2.13.2-9ubuntu6.1) disco-proposed; urgency=medium

  * lp1820068.patch: don't skip read cache when options are set (LP: #1820068)
  * reenable ubuntu/parser-conf-no-expr-simplify.patch

 -- Jamie Strandboge <email address hidden> Thu, 06 Jun 2019 21:04:34 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for apparmor has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.