expr-simplify optimization slows click/snap policy compilation

[Jamie Strandboge]

AppArmor has several optimization options that can be used to help speed up policy compiles for certain types of policy. Currently, we are using expr tree simplification option by default, which has dramatic affects on policy compiles for the evince profile. However, with click profiles not using expr tree simplification (ie, adding the '-O no-expr-simplify' option) can improve click policy generation by 44% (375 vs 210 seconds).

On Krillin, the difference is even more substantial: 636 vs 233 seconds (63%).

Short term for rtm is to to use '-O no-expr-simplify' when compiling policy in /var/lib/apparmor/profiles but leave /etc/apparmor.d alone. We can do the same with click-apparmor. Note: the fix for bug #1385947 must be included with this fix.

The long term fix is to adjust expr tree simplification to be more efficient (at least as fast as without) and drop the '-O no-expr-simplify' option.

Justification: apparmor policy recompilation is not expected to happen as part of the normal user experience (see bug #1350598 for a lot of detail) and it is expected to only happen on upgrades from 14.10 to 15.04 or to fix very serious apparmor or apparmor policy bugs. None of these bugs are currently scheduled for OTA. However, *if* we ever need to fix one of these, policy will have to be recompiled.

Choices:
1. do nothing for RTM since policy recompiles are expected to be rare, but do apply this change to 15.04. Policy is expected to be recompiled on upgrades to 15.04 and upgrades would use the new option
2. apply this change in OTA. This is problematic because this change alone will trigger a policy recompilation that would not otherwise be needed. Optionally, this change could accompany a severe bug fix

Risk:
The change consists of a small modification to the apparmor upstart job and a change to the arguments click-apparmor gives to apparmor_parser. The risk assessment is considered low because of the size of the change and the simple test case will immediately indicate if either were applied incorrectly.

Test case:
1. run aa-status | wc -l and note the result
2. install the new apparmor and click-apparmor packages and verify there are no errors during installation
3. reboot
4. run aa-status | wc -l and compare to '1'
5. run 'sudo start apparmor' and make sure it finishes in a few seconds

If they are the same, it indicates the upstart job is properly loading the profiles generated by click apparmor.

While these changes may occur separately, landing them at the same time along with a regenerated custom tarball (for preinstalled policy) will reduce policy recompiles.

[Launchpad Janitor (janitor)]

This bug was fixed in the package click-apparmor - 0.2.11.2

---------------
click-apparmor (0.2.11.2) utopic; urgency=medium

  * aa-clickhook: disable expr tree simplification when calling the parser
    since this considerably slows down click policy compiles (LP: #1383858)
 -- Jamie Strandboge <email address hidden> Wed, 22 Oct 2014 14:24:15 -0400

[Jamie Strandboge (jdstrand)]

Here is the debdiff for apparmor, in case anyone needs it:

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.8.98-0ubuntu3

---------------
apparmor (2.8.98-0ubuntu3) vivid; urgency=medium

  * debian/lib/apparmor/functions: disable expr tree simplification for
    /var/lib/apparmor/profiles (LP: #1383858)
  * parser-dont-skip-read-cache-with-optimizations.patch: don't skip read
    cache when specifying '-O' (LP: #1385947)
 -- Jamie Strandboge <email address hidden> Tue, 28 Oct 2014 17:41:08 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.8.96~2652-0ubuntu5.3

---------------
apparmor (2.8.96~2652-0ubuntu5.3) 14.09; urgency=medium

  * debian/lib/apparmor/functions: disable expr tree simplification for
    /var/lib/apparmor/profiles (LP: #1383858)
  * parser-dont-skip-read-cache-with-optimizations.patch: don't skip read
    cache when specifying '-O' (LP: #1385947)
  * 10-lp1371771.patch, 11-lp1371765.patch: refresh for the above
 -- Jamie Strandboge <email address hidden> Mon, 27 Oct 2014 07:13:42 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package click-apparmor - 0.2.11.2

---------------
click-apparmor (0.2.11.2) utopic; urgency=medium

  * aa-clickhook: disable expr tree simplification when calling the parser
    since this considerably slows down click policy compiles (LP: #1383858)
 -- Jamie Strandboge <email address hidden> Wed, 22 Oct 2014 14:24:15 -0400

[Ivan Baldo (ibaldo)]

Hello.
Can you guys take a look at this suggestion please?
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1830502/comments/18
Thanks!!!

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

This MR for apparmor_parser:

https://gitlab.com/apparmor/apparmor/-/merge_requests/711

helps quite a bit with accelerating the expression optimization, and actually makes it worth enabling them back.