Support for grub upgrades with bios+uefi bootloader targets

[Łukasz Zemczak]

[Impact]

There are multiple use cases which require both BIOS and UEFI bootloaders installed on a target image and to keep them both updated.
- cloud images on clouds that support both BIOS and UEFI boot in alternate instance types
- PC installs that should remain bootable in the face of firmware upgrades or reconfigurations

This currently doesn't work because 'grub-install' selects its install target based on which of grub-pc or grub-efi-amd64 is installed.

In cosmic we have introduced a --auto-nvram grub-install option that automatically determines if we're running with NVRAM access or not and if yes, updates the NVRAM contents. This allows such dual BIOS-UEFI bootloader setups to work. Same changes are required to be backported to bionic for our cloud images.

[Test Case]

Basic grub2 grub-install test:
 * Boot up a bionic system in UEFI mode.
 * Upgrade grub2-common to the version in -proposed.
 * Run `grub-install --target=x86_64-efi --auto-nvram` and make sure it succeeds.
 * Prepare a system with an UEFI installed system that can be booted into in legacy BIOS mode.
 * Boot up the UEFI-installed bionic system in legacy BIOS mode.
 * Upgrade grub2-common to the version in -proposed.
 * Run `grub-install --target=x86_64-efi --auto-nvram` and make sure it succeeds (actually not doing anything).

Install test for UEFI (repeat for both server-live, server and desktop):
 * Download the latest bionic -proposed-enabled image.
 * Make sure the image includes the -proposed version of grub2, grub2-signed, shim-signed and grub-installer (and/or ubiquity).
 * Install the system normally on an EFI system.
 * Reboot and make sure the system is bootable.

Install test for legacy BIOS (repeat for both server-live, server and desktop):
 * Download the latest bionic -proposed-enabled image.
 * Make sure the image includes the -proposed version of grub2, grub2-signed, shim-signed and grub-installer (and/or ubiquity).
 * Install the system normally on a BIOS system.
 * Reboot and make sure the system is bootable.

TODO: Add cloud image testing.

[Regression Potential]

The backport introduces a change in the dependency chain for grub which, in some cases, can lead to systems loosing their ability to boot. Basically the symptoms to look for is the inability of booting the installed system on EFI or BIOS. A lot of testing and dogfooding will be required to make sure no installation-case has been broken by this.

[Łukasz Zemczak (sil2100)]

Performing some test builds before submitting packages to the SRU queue: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3308/+packages

[Brian Murray (brian-murray)]

It looks like this was fixed in cosmic but referenced a different bug number.
grub-installer (1.128ubuntu9) cosmic; urgency=medium

  * grub-installer: install grub-pc for EFI setups and make sure we're
    not purging it earlier in case it got installed. It's needed to make sure
    the right maintainer scripts are run and the ESP populated. (LP: #1775743)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Tue, 19 Jun 2018 10:25:14 +0200

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted grub-installer into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub-installer/1.128ubuntu8.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

This too was fixed in cosmic.

shim-signed (1.36) cosmic; urgency=medium

  * debian/shim-signed.postinst: use --auto-nvram with grub-install in case
    we're installing on a NVRAM-unavailable platform.
  * debian/control: bump the dependency for grub2-common to make sure
    grub-install supports --auto-nvram.
  * debian/control: switch the grub-efi-amd64-bin dependency to
    grub-efi-amd64-signed.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 06 Jun 2018 20:25:57 +0200

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted shim-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.34.9.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Also fixed in cosmic.grub2-signed (1.96) cosmic; urgency=medium

  * debian/control: switch the grub-efi-amd64 dependency of
    grub-efi-amd64-signed to grub-efi-amd64-bin.
  * debian/grub-efi-amd64-signed.postinst: invoke grub-install with
    --auto-nvram and pass the x86_64-efi target to it, making sure we always
    install the right target.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 06 Jun 2018 20:09:57 +0200

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted grub2-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.93.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

This too was fixed in cosmic.
grub2 (2.02-2ubuntu9) cosmic; urgency=medium

  * debian/patches/add-an-auto-nvram-option-to-grub-install.patch: Add the
    --auto-nvram option to grub-install for auto-detecting NVRAM availability
    before attempting NVRAM updates.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Tue, 05 Jun 2018 00:34:38 +0200

[Brian Murray (brian-murray)]

Hello Łukasz, or anyone else affected,

Accepted grub2 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.02-2ubuntu8.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Uploaded ubiquity with the new -proposed packages updated. That is necessary for the fixes to work on desktop.

[Steve Langasek (vorlon)]

Hello Łukasz, or anyone else affected,

Accepted ubiquity into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/18.04.14.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Phillip Susi (psusi)]

Why would you want to install grub-efi if you can't update the EFI boot catalog? Without that, it can't be booted in EFI mode.

[Steve Langasek (vorlon)]

On Mon, Jul 02, 2018 at 12:14:40AM -0000, Phillip Susi wrote:
> Why would you want to install grub-efi if you can't update the EFI boot
> catalog? Without that, it can't be booted in EFI mode.

Because with shim's 'fallback' implementation, a system that could not
update the boot catalog in nvram because it was not booted under UEFI
initially still stands a chance of auto-configuring itself to be able to
boot Ubuntu, via selecting the disk for boot from the firmware boot
selector.

[Łukasz Zemczak (sil2100)]

Verification for bionic, basic grub2 grub-install test (using bionic ubuntu-server daily image 20180704):

I had a legacy BIOS system on a virtual HDD. Using kvm, I booted the bionic daily image in UEFI mode and installed a new Ubuntu system next to it (with the ESP created etc.). I have then booted the UEFI system and ran `grub-install --target=x86_64-efi --auto-nvram` - the command returned with return value of 0.
Then I switched to legacy BIOS mode and booted into the UEFI-installed system. I have run the `grub-install --target=x86_64-efi --auto-nvram` command there again and the command also returned with the return value of 0 (success).

Proceeding with installation tests.

[Łukasz Zemczak (sil2100)]

ubuntu-server daily (d-i) (20180704):

Confirmed in the image build logs that the image contains the right versions of grub2, grub2-signed, shim-signed and grub-installer [1].

 UEFI install:
 * Performed a full-disk installation that resulted in a bootable system.
 BIOS install:
 * Performed a full-disk installation that resulted in a bootable system.

ubuntu-desktop daily-live (ubiquity) (20180704):

Confirmed in the image build logs that the image contains the right versions of grub2, grub2-signed, shim-signed and ubiquity [2]

 UEFI install:
 * Performed a full-disk installation that resulted in a bootable system.
 BIOS install:
 * Performed a full-disk installation that resulted in a bootable system.

[1] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-server/bionic/daily-20180704.log
[2] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu/bionic/daily-live-20180704.log

[Łukasz Zemczak (sil2100)]

I am unable to verify the ubuntu-server subiquity based images as the installation currently is failing. It looks to me unrelated to the proposed grub changes as it's a problem with getting the right kernel installed - I'll poke the subiquity team about this.

[Steve Langasek (vorlon)]

While the complete verification of this is not yet done with respect to the server live image, the ubiquity+grub-installer pieces (affecting d-i and ubiquity install paths) have been verified, and there is another grub-installer SRU waiting in the unapproved queue; so I'm going to go ahead with releasing these two packages since they don't need to go in at the same time as shim+grub2. The others can be released on Monday after verification is completed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub-installer - 1.128ubuntu8.18.04.1

---------------
grub-installer (1.128ubuntu8.18.04.1) bionic; urgency=medium

  * grub-installer: install grub-pc for EFI setups and make sure we're
    not purging it earlier in case it got installed. It's needed to make sure
    the right maintainer scripts are run and the ESP populated.
    Required as part of the dependency-chain switch for dual EFI/BIOS support.
    (LP: #1778848)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 27 Jun 2018 10:27:18 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 18.04.14.3

---------------
ubiquity (18.04.14.3) bionic; urgency=medium

  * Automatic update of included source packages: grub-installer
    1.128ubuntu8.18.04.1, partman-auto 134ubuntu8.1, partman-efi
    71ubuntu2.1. (LP: #1766945, LP: #1778848)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Fri, 29 Jun 2018 09:47:18 +0200

[Łukasz Zemczak (sil2100)]

ubuntu-server daily-live (subiquity) (20180709):

Confirmed in the image build logs that the image contains the right versions of grub2, grub2-signed and shim-signed [1]. The previous issues with subiquity were related to the kernel binaries being stuck in NEW (so unrelated). By the time of 20180709 everything was good and testing could be performed.

 UEFI install:
 * Performed a full-disk installation that resulted in a bootable system.
 BIOS install:
 * Performed a full-disk installation that resulted in a bootable system.

[1] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-server/bionic/daily-live-20180709.log

This basically concludes the testing from my side.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.02-2ubuntu8.1

---------------
grub2 (2.02-2ubuntu8.1) bionic; urgency=medium

  * debian/patches/add-an-auto-nvram-option-to-grub-install.patch: Add the
    --auto-nvram option to grub-install for auto-detecting NVRAM availability
    before attempting NVRAM updates. (LP: #1778848)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Tue, 05 Jun 2018 00:34:38 +0200

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for grub2 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2-signed - 1.93.1

---------------
grub2-signed (1.93.1) bionic; urgency=medium

  * debian/control: switch the grub-efi-amd64 dependency of
    grub-efi-amd64-signed to grub-efi-amd64-bin. (LP: #1778848)
  * debian/grub-efi-amd64-signed.postinst: invoke grub-install with
    --auto-nvram and pass the x86_64-efi target to it, making sure we always
    install the right target. (LP: #1778848)
  * debian/control: rebuild against grub2 2.02-2ubuntu8.1.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 27 Jun 2018 09:01:13 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package shim-signed - 1.34.9.2

---------------
shim-signed (1.34.9.2) bionic; urgency=medium

  * debian/shim-signed.postinst: use --auto-nvram with grub-install in case
    we're installing on a NVRAM-unavailable platform. (LP: #1778848)
  * debian/control: bump the dependency for grub2-common to make sure
    grub-install supports --auto-nvram. (LP: #1778848)
  * debian/control: switch the grub-efi-amd64-bin dependency to
    grub-efi-amd64-signed. (LP: #1778848)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 27 Jun 2018 10:14:24 +0200

[Phillip Susi (psusi)]

On 07/02/2018 12:51 PM, Steve Langasek wrote:
> On Mon, Jul 02, 2018 at 12:14:40AM -0000, Phillip Susi wrote:
>> Why would you want to install grub-efi if you can't update the EFI boot
>> catalog? Without that, it can't be booted in EFI mode.
> Because with shim's 'fallback' implementation, a system that could not
> update the boot catalog in nvram because it was not booted under UEFI
> initially still stands a chance of auto-configuring itself to be able to
> boot Ubuntu, via selecting the disk for boot from the firmware boot
> selector.
>
You mean we are now installing a "removable media" boot loader in
EFI/BOOT that chainloads the one in EFI/Ubuntu?

[Steve Langasek (vorlon)]

On Tue, Jul 17, 2018 at 07:15:06PM -0000, Phillip Susi wrote:
> On 07/02/2018 12:51 PM, Steve Langasek wrote:
> > On Mon, Jul 02, 2018 at 12:14:40AM -0000, Phillip Susi wrote:
> >> Why would you want to install grub-efi if you can't update the EFI boot
> >> catalog? Without that, it can't be booted in EFI mode.
> > Because with shim's 'fallback' implementation, a system that could not
> > update the boot catalog in nvram because it was not booted under UEFI
> > initially still stands a chance of auto-configuring itself to be able to
> > boot Ubuntu, via selecting the disk for boot from the firmware boot
> > selector.

> You mean we are now installing a "removable media" boot loader in
> EFI/BOOT that chainloads the one in EFI/Ubuntu?

The "removable media" bootloader does not chainload the one in EFI\Ubuntu;
it chainloads fbx64.efi, whose sole function is to (re)populate the nvram
boot settings from the set of BOOT.CSV files it discovers on disk.

https://github.com/rhboot/shim/blob/master/README.fallback

So even if the we cannot populate nvram at install time (e.g. because the
system is not booted in EFI mode at that point), it is useful to initialize
and populate an ESP for future EFI compatibility.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
<email address hidden> <email address hidden>