backport/sync UEFI, Secure Boot support

[Mathieu Trudel-Lapierre]

[Impact]
Since the implementation of UEFI Secure Boot in Ubuntu, there has been a large number of changes to the EFI patchset, handled "upstream" at https://github.com/vathpela/grub2-fedora/tree/sb.

This SRU is handled as a wholesale "sync" with a known set of patches rather than individual cherry-picks given the high risk in cherry-picking individual changes; we do not want to risk subtly breaking Secure Boot support or introducing a security issue due to using different sets of patches across our currently supported releases. Using a common set of patches across releases and making sure we're in sync with "upstream" for that particular section of the grub2 codebase (specifically, UEFI/SB support is typically outside the GNU GRUB tree) allows us to make sure UEFI Secure Boot remains supportable and that potential security issues are easy to fix quickly given the complexity of the codebase.

This is a complex set of enablement patches; most of them will be fairly straightforward backports, but there are a few known warts:

 * The included patches are based on grub2 2.02~beta3; as such, some patches require extra backporting effort of other pieces of the loader code down to releases that do not yet include 2.02~beta3 code.

[Test Case]
The desktop, server, and alternate install images should all boot and install on an SB-enabled system. I would recommend testing installations from both a CD and a USB stick. After each installation, validate that Secure Boot is enabled by checking /sys/firmware/efi/efivars/SecureBoot-*, as well as /sys/firmware/efi/efivars/Mok* variables (for the cases where shim validation may be disabled).

Tests should include:
- booting with Secure Boot enabled
- booting with Secure Boot enabled, but shim validation disabled
- booting with Secure Boot disabled, but still in EFI mode

[Regression Potential]
Check that non-SB installations of all these images still work. For this, it is sufficient to test with either a CD or a USB stick, but not necessarily both.

[Mathieu Trudel-Lapierre (cyphermox)]

Updating status to clarify that this is already uploaded and included in the artful release.

[Chris Halse Rogers (raof)]

Just to be clear, what is the purpose of this backport? As you know, “upstream has done more work” isn't usually justification for an SRU :)

Presumably this is expected to fix bugs and/or support new systems? Could you give a brief run-down of what this fixes/newly supports?

[Mathieu Trudel-Lapierre (cyphermox)]

I've updated the description to make the rationale clearer. This is a general backport of the patchset coming from "upstream" (in this case, being the "UEFI community" instead of GNU GRUB, and personified in this git tree from fedora), which include changes such as:

 - general cleanup and fixes (memory usage, etc.)
 - load arm with SB enabled
 - fixing a race in EFI validation (verifying Secure Boot signature for a kernel)
 - allow chainloading including the device part of the EFI boot path (chainloading across drives, for example)
 - honour Secure Boot in the chainloader (verify via Shim, not just EFI Boot Services)
 - avoid loading modules not permissible in Secure Boot
 - fixes for PE section alignment (mostly related to chainloading the Windows bootloader)
 - properly handle Secure Boot state when loading images (behaving correctly when Secure Boot validation in shim is disabled; correctly interpreting the result of shim's Secure Boot validation failing in the cases where SB is disabled in firmware vs. when it is disabled in shim or when not booting through shim)

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.02~beta2-36ubuntu3.12 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2 into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.02~beta2-36ubuntu11.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2 into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.02~beta3-4ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2-signed into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.80.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2-signed into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.74.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello Mathieu, or anyone else affected,

Accepted grub2-signed into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.66.12 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Ubuntu Foundations Team Bug Bot (crichton)]

As a part of the Stable Release Updates quality process a search for Launchpad bug reports using the version of grub2-signed from zesty-proposed was performed and bug 1699790 was found. Please investigate this bug report to ensure that a regression will not be created by this SRU. In the event that this is not a regression remove the "verification-failed" tag from this bug report and add the tag "bot-stop-nagging" to bug 1699790 (not this bug). Thanks!

[Ubuntu Foundations Team Bug Bot (crichton)]

As a part of the Stable Release Updates quality process a search for Launchpad bug reports using the version of grub2 from zesty-proposed was performed and bug 1700132 was found. Please investigate that bug report to ensure that a regression will not be created by this SRU. In the event that this is not a regression remove the "verification-failed" tag from this bug report and add the tag "bot-stop-nagging" to bug 1700132 (not this bug). Thanks!

[Łukasz Zemczak (sil2100)]

As part of a recent change in the Stable Release Update verification policy we would like to inform that for a bug to be considered verified for a given release a verification-done-$RELEASE tag needs to be added to the bug where $RELEASE is the name of the series the package that was tested (e.g. verification-done-xenial). Please note that the global 'verification-done' tag can no longer be used for this purpose.

Thank you!

[Mathieu Trudel-Lapierre (cyphermox)]

Verification-done for xenial with grub2 2.02~beta2-36ubuntu3.12 and grub2-signed 1.66.12:

Booting behaves correctly with Secure Boot enabled or disabled; and when Secure Boot is enabled but shim validation is disabled. Booting in the chainload case could not be tested (I do not have a Windows key to test with, will attempt to resolve this situation).

[Mathieu Trudel-Lapierre (cyphermox)]

Verification-done for zesty with grub2 2.02~beta3-4ubuntu2.2 and grub2-signed 1.80.2:

Booting in insecure mode, with Secure Boot enabled and with SB disabled (but UEFI enabled) have been tested and all work as expected.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.02~beta3-4ubuntu2.2

---------------
grub2 (2.02~beta3-4ubuntu2.2) zesty; urgency=medium

  * debian/patches: Rework linuxefi/SecureBoot support and sync with upstream
    SB patch set: (LP: #1696599)
    - linuxefi_arm_sb_support.patch: add Secure Boot support for arm for its
      chainloader.
    - linuxefi_fix_validation_race.patch: Fix a race in validating images.
    - linuxefi_chainloader_path.patch: honor the starting path for grub, so
      images do not need to be started from $root.
    - linuxefi_chainloader_sb.patch: Fix some more issues in chainloader use
      when Secure Boot is enabled.
    - linuxefi_loaders_enforce_sb.patch: Enforce Secure Boot policy for all
      loaders: don't load the commands when Secure Boot is enabled.
    - linuxefi_re-enable_linux_cmd.patch: Since we rely on the linux and
      initrd commands to automatically hand-off to linuxefi/initrdefi; re-
      enable the linux loader.
    - linuxefi_chainloader_pe_fixes.patch: PE parsing fixes for chainloading
      "special" PE images, such as Windows'.
    - linuxefi_rework_non-sb_cases.patch: rework cases where Secure Boot is
      disabled or shim validation is disabled so loading works as EFI binaries
      when it is supposed to.
    - Removed linuxefi_require_shim.patch; superseded by the above.
      (LP: #1689687)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 14 Jun 2017 14:44:48 -0400

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for grub2 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2-signed - 1.80.2

---------------
grub2-signed (1.80.2) zesty; urgency=medium

  * Rebuild against grub2 2.02~beta3-4ubuntu2.2. (LP: #1696599)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 14 Jun 2017 14:46:59 -0400

[Mathieu Trudel-Lapierre (cyphermox)]

Finally got to verifying this for chainloading as well, on both xenial and zesty -- marking xenial as verification-done now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.02~beta2-36ubuntu3.12

---------------
grub2 (2.02~beta2-36ubuntu3.12) xenial; urgency=medium

  * debian/patches: Rework linuxefi/SecureBoot support and sync with upstream
    SB patch set: (LP: #1696599)
    - linuxefi_backport_arm64.patch: backport basic arm64 chainload/linux
      command support from 17.04.
    - linuxefi_arm_sb_support.patch: add Secure Boot support for arm for its
      chainloader.
    - linuxefi_fix_validation_race.patch: Fix a race in validating images.
    - linuxefi_chainloader_path.patch: honor the starting path for grub, so
      images do not need to be started from $root.
    - linuxefi_chainloader_sb.patch: Fix some more issues in chainloader use
      when Secure Boot is enabled.
    - linuxefi_loaders_enforce_sb.patch: Enforce Secure Boot policy for all
      loaders: don't load the commands when Secure Boot is enabled.
    - linuxefi_re-enable_linux_cmd.patch: Since we rely on the linux and
      initrd commands to automatically hand-off to linuxefi/initrdefi; re-
      enable the linux loader.
    - linuxefi_chainloader_pe_fixes.patch: PE parsing fixes for chainloading
      "special" PE images, such as Windows'.
    - linuxefi_rework_non-sb_cases.patch: rework cases where Secure Boot is
      disabled or shim validation is disabled so loading works as EFI binaries
      when it is supposed to.
    - Removed linuxefi_require_shim.patch; superseded by the above.
      (LP: #1689687)
  * debian/patches/git_tsc_use_alt_delay_sources_d43a5ee6.patch: refreshed.
  * debian/patches/arm64-set-correct-length-of-device-path-end-entry.patch:
    dropped; included in linuxefi_backport_arm64.patch.

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 08 Jun 2017 10:16:17 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2-signed - 1.66.12

---------------
grub2-signed (1.66.12) xenial; urgency=medium

  * Rebuild against grub2 2.02~beta2-36ubuntu3.12. (LP: #1696599)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 14 Jun 2017 14:39:30 -0400

[Steve Langasek (vorlon)]

Superseded by LP: #1708245 for trusty.

[Mathieu Trudel-Lapierre (cyphermox)]

I think we'll need to backport the UEFI Secure Boot patches to trusty after all -- there's a large number of changes in them, but it seems better than attempting to adapt other patches (such as Windws 7/10 chainloading fixes with new shim, and memory truncation fixes). Having the patchset at the same level will make it easier to support Secure Boot on trusty. At least with the same patches we can quickly issue security fixes if there are any issues, as the code will be roughly the same as for other releases.

[Brian Murray (brian-murray)]

Hello Mathieu, or anyone else affected,

Accepted grub2 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.02~beta2-9ubuntu1.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Mathieu, or anyone else affected,

Accepted grub2-signed into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.34.18 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mathieu Trudel-Lapierre (cyphermox)]

Verification-done for trusty for grub2 and grub2-signed:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-====================================================-============-===============================================================================
ii grub-efi-amd64 2.02~beta2-9ubuntu1.16 amd64 GRand Unified Bootloader, version 2 (EFI-AMD64 version)
ii grub-efi-amd64-signed 1.34.18+2.02~beta2-9ubuntu1.16 amd64 GRand Unified Bootloader, version 2 (EFI-AMD64 version, signed)

Chainloading works correctly now, and normal loading of a Linux kernel to boot to Ubuntu also works correctly. Loading an unsigned kernel is still allowed, but debug mode does show the expected verification behavior happening at boot.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.02~beta2-9ubuntu1.16

---------------
grub2 (2.02~beta2-9ubuntu1.16) trusty; urgency=medium

  [ Ivan Hu ]
  * debian/patches/0001-i386-linux-Add-support-for-ext_lfb_base.patch:
    Add support for ext_lfb_base. (LP: #1785033)

  [ dann frazier ]
  * Add grub2/update_nvram template to allow users to disable NVRAM
    updates during package upgrades (LP: #1642298).

  [ Mathieu Trudel-Lapierre ]
  * debian/patches: Rework linuxefi/SecureBoot support and sync with upstream
    SB patch set: (LP: #1696599)
    - linuxefi_backport_arm64.patch: backport basic arm64 chainload/linux
      command support from 17.04.
    - linuxefi_arm_sb_support.patch: add Secure Boot support for arm for its
      chainloader.
    - linuxefi_fix_validation_race.patch: Fix a race in validating images.
    - linuxefi_chainloader_path.patch: honor the starting path for grub, so
      images do not need to be started from $root.
    - linuxefi_chainloader_sb.patch: Fix some more issues in chainloader use
      when Secure Boot is enabled.
    - linuxefi_loaders_enforce_sb.patch: Enforce Secure Boot policy for all
      loaders: don't load the commands when Secure Boot is enabled.
    - linuxefi_re-enable_linux_cmd.patch: Since we rely on the linux and
      initrd commands to automatically hand-off to linuxefi/initrdefi; re-
      enable the linux loader.
    - linuxefi_chainloader_pe_fixes.patch: PE parsing fixes for chainloading
      "special" PE images, such as Windows'.
    - linuxefi_rework_non-sb_cases.patch: rework cases where Secure Boot is
      disabled or shim validation is disabled so loading works as EFI binaries
      when it is supposed to.
    - Removed linuxefi_require_shim.patch; superseded by the above.
    - Removed linuxefi_amd64_only.patch; superseded by the above.
    - Refreshed patches.
  * debian/rules: disable the use of -Werror while building grub; the EFI
    patches have subtle cases which trip it up unnecessarily.
  * debian/patches/arm64-set-correct-length-of-device-path-end-entry.patch:
    dropped; included in linuxefi_backport_arm64.patch.
  * debian/patches/linuxefi_fix_relocate_coff.patch: fix typo in
    relocate_coff() causing issues with relocation of code in chainload.
    (LP: #1792575)
  * debian/patches/linuxefi_truncate_overlong_relocs.patch: The Windows
    7 bootloader has inconsistent headers; truncate to the smaller, correct
    size to fix chainloading Windows 7. (LP: #1792575)

 -- Mathieu Trudel-Lapierre <email address hidden> Tue, 08 Jan 2019 12:36:49 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2-signed - 1.34.18

---------------
grub2-signed (1.34.18) trusty; urgency=medium

  * Rebuild against grub-efi-amd64 2.02~beta2-9ubuntu1.16
    (LP: #1785033) (LP: #1642298) (LP: #1696599) (LP: #1792575)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 09 Jan 2019 09:11:55 -0500