hook into system-image updates to precompile policy prior to reboot

Bug #1385410 reported by Jamie Strandboge
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
click-apparmor (Ubuntu)
Confirmed
High
Unassigned
unity8 (Ubuntu)
New
Undecided
Unassigned

Bug Description

Occasionally users will receive and OTA update that requires apparmor policy to be recompiled. Recompiling apparmor policy can take quite a bit of time (minutes) depending on how many applications the user has installed. While this is only expected to happen on major OTA OS upgrades (eg, 14.10 to 15.04), it is possible it could happen at other times. This would improve the user experience for developers considerably since policy recompiles can be relatively frequent when running the development release.

<https://lists.launchpad.net/ubuntu-phone/msg13291.html>: "On start apparmor profiles need to be regenerated which means the first boot after the upgrade might take long depending on the number of installed applications (even up to 30 minutes). Currently there will be no visual indication of the profile regeneration, so please be patient and wait until you see the UI appearing."

To improve the user experience, we should detect and recompile apparmor profiles prior to reboot but after system-image updates has downloaded the new update. This always for the possibility of using a progress meter when compiling policy (which we currently cannot).

<https://wiki.ubuntu.com/SoftwareUpdates#restart-and-install>: "Therefore, every step of the system update should show a progress bar, that fills once across all the steps ... Subtask Recompiling apparmor profiles; Allocation 45%; Resulting range 0~45%"

See also bug 1350598, about caching compiled policies more often.

Revision history for this message
Michał Sawicz (saviq) wrote :

Would it be at all possible to do this in recovery?

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Not really-- it was determined at the sprint that the recovery doesn't have enough available to do this properly.

Revision history for this message
Michał Sawicz (saviq) wrote :

I'm feeling the best we could do is actually doing it on first boot after OTA, but make sure that we show a different splash screen while this is happening.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

This happens during the 'Google' logo on boot after flash and I'm told there is no way to provide progress during this point. For traditional apt based systems we will be employing a similar technique as outlined in this bug-- during postinst we compile policy as needed so the user is free to use the system while this is happening.

For system-image based systems, why is it not acceptable to compile the policy prior to reboot? It provides a lot of flexibility (we can choose to do this in the background if the image is already downloaded), it can provide progress feedback easily and it allow the user to use another app while the compile is happening.

Revision history for this message
Michał Sawicz (saviq) wrote :

I'm not actually sure how you can compile them before reboot (so before the update was applied), but if you say that's possible, I'm sure we can think of something. We should probably stop the session straight away and display a progress spinner akin to the one in recovery. I just want to try and limit the number of user-visible "steps" an update is taking, is all.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

We figured out a way to be able to do it before reboot, and this is something that is needed for systemd and helps usability of traditional apt systems in addition to system-image systems.

The idea is there are no extra steps for the user to go through, but yes the idea is there would be a progress meter after the user presses 'Install and reboot' (though even that isn't strictly required but since it could take awhile, it seems to make sense).

Changed in ubuntu-ux:
assignee: nobody → Matthew Paul Thomas (mpt)
status: New → Triaged
summary: - hook into system-image updates to precompile policy prior to reboot
+ [system updates] hook into system-image updates to precompile policy
+ prior to reboot
Changed in ubuntu-ux:
importance: Undecided → Medium
Revision history for this message
Matthew Paul Thomas (mpt) wrote : Re: [system updates] hook into system-image updates to precompile policy prior to reboot

The spec already includes a determinate progress bar when installing system updates. Unfortunately, this isn't implemented yet. <https://wiki.ubuntu.com/SoftwareUpdates#installing-mobile-system>

According to bug 1359344, the update itself happens after the restart. So if the precompilation happens after the restart too, that would be fine. If the precompilation happens before the restart, that would be a bit worse, but not terrible: the progress bar could advance from 0 to 40% (for example) for the precompilation, then the device could restart, then (assuming that the restart took 5% of the time) the progress bar could reappear, advancing from 45% to 100%. You wouldn't be able to do anything else during the installation, so that the device restarted in the middle would be merely an implementation detail.

That disappearance and reappearance of the progress bar would be worthwhile if it allowed for faster updates overall: for example, if the precompilation could begin, or even complete, before the rest of the system image downloaded or before you decided that now was a good time to restart.

So to design this, I need the answers to three questions:

A. Is it practical to use the device while precompilation is happening in the background?

B. What drawbacks, if any, are there from using the device after precompilation finishes, but before the device restarts?

C. When recompilation is required, what proportion of overall update installation time, on average, is taken up by (i) the precompilation (ii) the restart (iii) the flashing (iv) anything else? (Bonus points if you manage to relate (i) to the number or size of the policies.)

Revision history for this message
Matthew Paul Thomas (mpt) wrote :

I can't do any more on this until those three questions are answered.

Changed in ubuntu-ux:
status: Triaged → Incomplete
summary: - [system updates] hook into system-image updates to precompile policy
- prior to reboot
+ hook into system-image updates to precompile policy prior to reboot
Revision history for this message
Seth Arnold (seth-arnold) wrote :

I'm sorry this was overlooked for so long.

> A. Is it practical to use the device while precompilation is happening in the background?

It Depends. We have some options in how aggressively we use system resources, but probably even compiling one-at-a-time with a nice value will definitely feel like the system is being used. Probably using PIM functions will be fine, office productivity might show some lags, games are probably not going to work.

> B. What drawbacks, if any, are there from using the device after precompilation finishes, but before the device restarts?

I believe no drawbacks at all.

> C. When recompilation is required, what proportion of overall update installation time, on average, is taken up by (i) the precompilation (ii) the restart (iii) the flashing (iv) anything else? (Bonus points if you manage to relate (i) to the number or size of the policies.)

I'm sorry to not have numbers to give you, but my recollection of anecdotal evidence suggests it can be fifteen minutes or more. My recollection of reboot times is roughly one minute, and no idea about flashing.

On my core i7 laptop, apparmor_parser takes 2.2 seconds and 20 megabytes of memory to compile the evince profile, typically one of the more expensive desktop and server profiles. I believe it is a fair approximation of a touch/snappy profile.

I hope this helps.

Changed in ubuntu-ux:
status: Incomplete → Triaged
description: updated
Changed in ubuntu-ux:
importance: Medium → High
Changed in ubuntu-ux:
status: Triaged → In Progress
Revision history for this message
Matthew Paul Thomas (mpt) wrote :
Changed in ubuntu-ux:
status: In Progress → Fix Committed
description: updated
Revision history for this message
Matthew Paul Thomas (mpt) wrote :

In the absence of any data, other than the "fifteen minutes or more" and "even up to 30 minutes" statements, I've specified that recompiling AppArmor profiles should take up the first 45% of the progress bar. If we can be smarter about that, then great:

- An improvement would be to match the average time taken up by profile recompilation in real-world measurements.

- Even better would be to vary the proportion based on whether the new image is based on a different Ubuntu release (more allocation) or not (less allocation).

- Even better still would be to vary the proportion based on the precise number of profiles that need upgrading.

All of that, though, is dependent on engineers/QA doing the timing to measure those proportions.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Note that the "15 minutes or more" is no longer the case and has not been for a long time. AppArmor policy compilation for 150 profiles or so should be in the 2-3 minute range (obviously, we still need feedback for the user).

Revision history for this message
Jamie Strandboge (jdstrand) wrote :
Revision history for this message
Matthew Paul Thomas (mpt) wrote :

It’s good to have a more precise estimate of (i) recompilation time. If/when we also get measurements for (ii) how long a restart usually takes, and (iii) how long flashing usually takes, I’ll update the specified progress bar allocation.

no longer affects: ubuntu-ux
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.