support compressed kernels on arm64

[Jim Hull]

[Impact]
Ubuntu kernels >= hwe-y (4.8) will no longer boot on xgene/uboot systems. The image size appears to have outgrown the defined firmware region, and u-boot will error out with a checksum mismatch error. A solution for this is to start shipping a compressed kernel (Image.gz target). flash-kernel needs updating to detect a compressed kernel and set the appropriate uImage compression flag. Similarly, d-i needs updating because it does it's own uImage generation.

[Test Case]
Boot an hwe-y kernel on an xgene/uboot system (APM Mustang, HP ProLiant m400).

[Regression Risk]
The necessary code is in place and in-use for yakkety, and no issues have been discovered there. The SRU involves clean cherry-picks of that code for d-i & f-k. The kernel side needs to be modified to pull in the correct version of flash-kernel for xenial.

[dann frazier (dannf)]

One comment on the patch - it adds a new dependency on the "file" package. So, if this method is used, then we'll also need to add it to debian/control.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "flash_kernel_compress.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[dann frazier (dannf)]

Here's an updated patch that avoids the dependency on file, and also adds a test case.

[Launchpad Janitor (janitor)]

This bug was fixed in the package flash-kernel - 3.0~rc.4ubuntu64

---------------
flash-kernel (3.0~rc.4ubuntu64) yakkety; urgency=medium

  * Add support for gzip-compressed kernel images (LP: #1384955)

 -- dann frazier <email address hidden> Thu, 25 Aug 2016 12:39:19 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.8.0-11.12

---------------
linux (4.8.0-11.12) yakkety; urgency=low

  * change_hat is logging failures during expected hat probing (LP: #1615893)
    - SAUCE: apparmor: Fix auditing behavior for change_hat probing

  * deleted files outside of the namespace are not being treated as
    disconnected
    (LP: #1615892)
    - SAUCE: apparmor: deleted dentries can be disconnected

  * stacking to unconfined in a child namespace confuses mediation
    (LP: #1615890)
    - SAUCE: apparmor: special case unconfined when determining the mode

  * apparmor module parameters can be changed after the policy is locked
    (LP: #1615895)
    - SAUCE: apparmor: fix: parameters can be changed after policy is locked

  * AppArmor profile reloading causes an intermittent kernel BUG (LP:
    #1579135)
    - SAUCE: apparmor: fix vec_unique for vectors larger than 8

  * label vec reductions can result in reference labels instead of direct
    access
    to labels (LP: #1615889)
    - SAUCE: apparmor: reduction of vec to single entry is just that entry

  * profiles from different namespaces can block other namespaces from being
    able to load a profile (LP: #1615887)
    - SAUCE: apparmor: profiles in one ns can affect mediation in another ns

  * The label build for onexec when stacking is wrong (LP: #1615881)
    - SAUCE: apparmor: Fix label build for onexec stacking.

  * The inherit check for new to old label comparison for domain transitions
    is
    wrong (LP: #1615880)
    - SAUCE: apparmor: Fix new to old label comparison for domain transitions

  * warning stack trace while playing with apparmor namespaces (LP: #1593874)
    - SAUCE: apparmor: fix stack trace when removing namespace with profiles

  * __label_update proxy comparison test is wrong (LP: #1615878)
    - SAUCE: apparmor: Fix __label_update proxy comparison test

  * reading /sys/kernel/security/apparmor/profiles requires CAP_MAC_ADMIN
    (LP: #1560583)
    - SAUCE: apparmor: Allow ns_root processes to open profiles file
    - SAUCE: apparmor: Consult sysctl when reading profiles in a user ns

  * policy namespace stacking (LP: #1379535)
    - SAUCE: (no-up) apparmor: rebase of apparmor3.5-beta1 snapshot for 4.8
    - SAUCE: add a sysctl to enable unprivileged user ns AppArmor policy loading

  * Miscellaneous Ubuntu changes
    - [Debian] Dynamically determine linux udebs package name
    - [Debian] d-i -- fix dtb handling in new kernel-wedge form
    - SAUCE: apparmor: Fix FTBFS due to bad include path
    - SAUCE: apparmor: add data query support
    - [Config] Set CONFIG_SECURITY_APPARMOR_UNCONFINED_INIT=y

  * Miscellaneous upstream changes
    - fixup backout policy view capable for forward port
    - apparmor: fix: Rework the iter loop for label_update
    - apparmor: add more assertions for updates/merges to help catch errors
    - apparmor: Make pivot root transitions work with stacking
    - apparmor: convert delegating deleted files to mediate deleted files
    - apparmor: add missing parens. not a bug fix but highly recommended
    - apparmor: add a stack_version file to allow detection of bug fixes
    - apparmor: push path looku...

[Robie Basak (racb)]

Are there any other systems that use flash-kernel that need to be considered in the test plan and regression potential sections?

From a flash-kernel perspective, based on the patch it seems to me that the decision to use a compressed kernel is outside the package, so I don't see a regression risk there. But other systems should be considered when landing the change that actually causes kernel images to be compressed, I think.

[Robie Basak (racb)]

Hello Jim, or anyone else affected,

Accepted flash-kernel into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/flash-kernel/3.0~rc.4ubuntu62.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dann frazier (dannf)]

I verified this by:
 - MAAS deploying an HP ProLiant m400 system w/ xenial
 - enabling proposed
 - adding ppa:canonical-kernel-team/unstable (for next step)
 - installing linux-image-extra-4.8.0-37-generic 4.8.0-37.39~16.04.2
 - verified that the previous step caused flash-kernel from proposed to be installed
 - verified that the new kernel landed a compressed kernel image in /boot
 - rebooting
 - verified that i booted into the newly installed kernel

[Launchpad Janitor (janitor)]

This bug was fixed in the package flash-kernel - 3.0~rc.4ubuntu62.2

---------------
flash-kernel (3.0~rc.4ubuntu62.2) xenial; urgency=medium

  * Add support for gzip-compressed kernel images (LP: #1384955)

 -- dann frazier <email address hidden> Tue, 31 Jan 2017 11:21:47 -0700

[Adam Conrad (adconrad)]

The verification of the Stable Release Update for flash-kernel has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[dann frazier (dannf)]

On Feb 8, 2017 08:06, "Robie Basak" <email address hidden> wrote:

Are there any other systems that use flash-kernel that need to be
considered in the test plan and regression potential sections?

>From a flash-kernel perspective, based on the patch it seems to me that
the decision to use a compressed kernel is outside the package, so I
don't see a regression risk there. But other systems should be
considered when landing the change that actually causes kernel images to
be compressed, I think.

X-Gene is the only arm64 SoC that I'm aware of that uses flash-kernel and
the Ubuntu generic kernel. The switch to compressed kernels was specific to
the Ubuntu arm64/generic kernel.

WRT the change in the kernel package to make images compressed, it has been
regression tested across the non-flash-kernel (i.e. UEFI) servers (GRUB
handles the decompression). This change is also restricted to the hwe
kernel in xenial, so it has no regression risk for those running the GA
kernel.

Both the kernel and flash-kernel changes were shipped in yakkety, and I am
unaware of any outstanding regressions. There was some initial fallout then
for d-i and maas-images, which generate uImages on their own, but those
have been resolved within those projects.

   -dann

[Launchpad Janitor (janitor)]

This bug was fixed in the package debian-installer - 20101020ubuntu451.10

---------------
debian-installer (20101020ubuntu451.10) xenial; urgency=medium

  [ dann frazier ]
  * arm64/xgene-uboot: Set the correct compression flag when generating
    a uImage of a gzip'd kernel. LP: #1384955.
  * Tell GRUB to load the gzio module on arm64 now that we're booting
    a compressed kernel. LP: #1632473.

  [ Adam Conrad ]
  * Add HWE variants for all architectures except powerpc (LP: #1662712)
  * Move master kernels to 4.4.0-62.
  * Rebuild to pick up new components (LP: #1572620, #1558271)

 -- Adam Conrad <email address hidden> Tue, 07 Feb 2017 07:13:40 -0700