shim-signed 15.4 does not boot on EFI 1.10 systems

[Balint Reczey]

The latest update on Hirsute made the MacBook Air from 2012 unbootable.

It could be recovered by booting a 20.04 live CD, downloading 20.04's shim package and overwriting the files in EFI/ubuntu and EFI/BOOT with the files shipped in the shim package.

Machines Affected (not comprehensive):
MacBookPro8,2
MacBookAir5,2

[Test case]
For a system that has applied the workaround from https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1925010/comments/14 and is bootable:

- install shim-signed from hirsute-proposed
- reboot
- verify that the system now boots without workaround

This test case applies only to hirsute. For all other series, there has been no regression, and because this is the same binary across all series, no other SRU testing is required.

[Dimitri John Ledkov (xnox)]

MBA 5,2 is https://support.apple.com/kb/SP670?locale=en_GB aka MacBook Air (13-inch, Mid 2012)

[Dimitri John Ledkov (xnox)]

Can you please check that the machine boots with just grub without shim.

Aka, replace /EFI/Boot/BOOTX64.efi & /ef/ubuntu/shimx64.efi files wtih /efi/ubuntu/grubx64.efi => this could possibly be a workaround, given that secureboot is not possible on Mac platforms.

[Dimitri John Ledkov (xnox)]

Upstream recommends to try https://github.com/rhboot/shim/pull/364/files patch. I'll prepare an upload of that into a PPA.

[Balint Reczey (rbalint)]

Replacing files as proposed in #2 works and makes the system bootable.

[Andreas Hasenack (ahasenack)]

Comment #2 also fixed this issue for my T420 laptop.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas Hasenack (ahasenack)]

More details, it's a T420 in uefi mode, but not secure boot. I was running groovy, and just upgraded to hirsute with do-release-upgrade -d. It just wouldn't boot from the disk anymore after that. Black screen, no disk activity, no ctrl-alt-del response. I always had to power cycle it.

[Andreas Hasenack (ahasenack)]

I moved the T420 issue to https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1925064

[satmandu (satadru-umich)]

Also got hit with this on my MacBookAir4,2 machine.

[satmandu (satadru-umich)]

Comment #2 also fixed this issue for my MacBookAir4,2 machine.

[Paul Ashbrook (ashbrook)]

Seems that my late-2012 Mac Mini is also suffering the same problem. I'll try the fixes as described above later on.

[Angus Fox (angusf)]

FWIW Upgrading worked fine, system boots normally and everything works apart from a Failed to start MokListXRT, out of resources message before the window manager loads.

This is on a Thinkpad with EFI v2.31 by Lenovo

[kjelderg (kjelderg)]

This was a problem for me with kubuntu 21.04 as well on a MacBookPro 11,2. The workaround did the trick. Exact workaround command follows:

/boot/efi/EFI# cp -b ubuntu/grubx64.efi ubuntu/shimx64.efi
/boot/efi/EFI# cp -b ubunt/grubx64.efi BOOT/BOOTX64.EFI

This works immediately.

[MikeMecanic (xyz-t)]

I have the same issue with secure boot enabled. Unsigned kernel 5.12-rc7+8 won't boot unless secure boot is disabled: invalid signature. Mok manager is not responding. ThinkPad Kubuntu or Ubuntu 21.04.

[Steve Langasek (vorlon)]

That is not the same issue. Please file a separate bug report.

[Isaac Cohen (icohen2000)]

Is this bug related, by any chance to this patch (https://github.com/rhboot/shim/pull/364)?

[Julian Andres Klode (juliank)]

Yes, it's exactly the same.

[Isaac Cohen (icohen2000)]

So couldn't we just put that version of shim into Ubuntu 21.04, or do we have to wait for the maintainers to do a release first? (Just curious about this.)

[Steve Langasek (vorlon)]

The Ubuntu shim binaries must be signed by microsoft in order to be useable on SecureBoot-enabled systems. We are currently waiting for these binaries to work through the process.

[T Jeske (t-jeske)]

Yes, and why can't we use the old binaries that worked before?

[Steve Langasek (vorlon)]

> Yes, and why can't we use the old binaries that worked before?

That's exactly what we're recommending you do by not upgrading to Ubuntu 21.04 until a newer fixed version is available.

We did not roll back shim in hirsute to the previous version because that version is expected to be revoked by Microsoft during the life cycle of Ubuntu 21.04.

[Julian Andres Klode (juliank)]

FWIW, "during the life cycle" = "today" - the revocations for the old shims have been published today: https://uefi.org/revocationlistfile

[Richard J Uschold (gilliganu1)]

This bug seems quite similar to the one I reported:
Ubuntu 20.04 LTS Fails to boot on Lenovo Flex 3-1480 laptop: Bug #1925710

As my bug explains, my 2015 Flex 3-1480 has a similar boot issue problem, though my 2019 Acer E15 Aspire E5-576 boots just fine.

Anyone think these are the same bug?

[Steve Langasek (vorlon)]

> Anyone think these are the same bug?

No. This bug only affects very old EFI implementations (over a decade old).
To my knowledge, only Apple ever shipped hardware using such
implementations. Certainly, any Lenovo systems from 2015 were shipping EFI
2.0.

[HouseSessions (koen-bulcke)]

For your information, I have an HP EliteBook 840 G2, I did a fresh install of 21.04 and had the same issue, it uses an "EFI v2.31 by HPQ".

I used following work around which worked.

What made the trick was the following changes in the BIOS Setup (after computer start push F10 - go to System Configuration - open Boot Options):
Check Customized Boot
Check off SecureBoot
Boot Mode: choose UEFI Hybrid or UEFI Native (I chose UEFI Native)
UEFI Boot Order: put Customized Boot to the top
Define Customized Boot Option: choose Add + put the setting: \EFI\ubuntu\grubx64.efi

Source: https://askubuntu.com/questions/244261/how-do-i-get-my-hp-laptop-to-boot-into-grub-from-my-new-efi-file

[Launchpad Janitor (janitor)]

This bug was fixed in the package shim - 15.4-0ubuntu2

---------------
shim (15.4-0ubuntu2) hirsute; urgency=medium

  [ Balint Reczey ]
  * Fix boot on EFI 1.10 machines, for example on some MacBooks (LP: #1925010)

  [ Dimitri John Ledkov ]
  * Fix kernel warning when allocating MOK table (LP: #1925139)
  * Fix booting with shim SBState disabled (LP: #1925140)

 -- Dimitri John Ledkov <email address hidden> Tue, 20 Apr 2021 15:24:29 +0100

[MikeMecanic (xyz-t)]

This new shim package fixes secure boot when booting an unsigned Kernel. Impish Indru Kubuntu 21.10. Thanks for the fix

2021-04-30 18:34:28 upgrade shim-signed:amd64 1.46+15.4-0ubuntu1 1.47+15.4-0ubuntu2

mokutil --sb
SecureBoot enabled
SecureBoot validation is disabled in shim
uname -srm
Linux 5.12.0-051200daily20210430-generic x86_64

[andschuster (andreas-schuster2)]

So the question remains, when the normal upgrade path from 20.10 to 21.04, that was stopped because of this bug, will be available.

[MikeMecanic (xyz-t)]

By now, the package is available in Impish-Proposed only. Tested today in Ubuntu 21.10 (20210501 ISO). All good for a second time. EFI 2.60 Lenovo/Ryzen 2700.

[Launchpad Janitor (janitor)]

This bug was fixed in the package shim-signed - 1.47

---------------
shim-signed (1.47) impish; urgency=medium

  [ Balint Reczey ]
  * Fix boot on EFI 1.10 machines, for example on some MacBooks (LP: #1925010)

  [ Dimitri John Ledkov ]
  * Fix kernel warning when allocating MOK table (LP: #1925139)
  * Fix booting with shim SBState disabled (LP: #1925140)
  * Use -Zxz compression, for compatibility with dpkg in older releases.
    LP: #1925673

shim-signed (1.46) hirsute; urgency=medium

  * New upstream release 15.4 LP: #1921134
  * Ship fb & mm from shim-signed package.
  * Remove shim-canonical-unsigned dependency, now provided by shim
    itself.
  * Generalize attaching externally supplied signatures, to aid building
    with embargoed or MS external signatures.

 -- Dimitri John Ledkov <email address hidden> Fri, 30 Apr 2021 10:46:25 +0100

[Łukasz Zemczak (sil2100)]

Hello Balint, or anyone else affected,

Accepted shim into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Balint, or anyone else affected,

Accepted shim-signed into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.47 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Alexander Browne (elcste)]

If I do the workaround described in comment #14 in order to get my MacBook to boot, and then install the shim-signed package from -proposed, will that be a valid test that the new package is working?

[Steve Langasek (vorlon)]

On Fri, May 07, 2021 at 03:24:15PM -0000, Alexander Browne wrote:
> If I do the workaround described in comment #14 in order to get my
> MacBook to boot, and then install the shim-signed package from
> -proposed, will that be a valid test that the new package is working?

Yes.

[Alexander Browne (elcste)]

I tested on a MacBookPro9,2 (13-Inch, Mid-2012).

0. I clean installed from the 21.04 release image (http://releases.ubuntu.com/21.04/ubuntu-21.04-desktop-amd64.iso). FWIW I made my usual choices

 - Minimal installation

 - Download updates…
 - Install third-party software…

 - Erase disk and install Ubuntu

1. Restarting after the install, the computer showed a gray screen and never booted up as per the original issue.

2. I booted again from the install media, opened the live session, did the workaround steps from #14 and booted into the Ubuntu 21.04 installation.

3. I enabled -proposed and ran `sudo apt install shim-signed` to upgrade that package to version 1.47+15.4-0ubuntu2.

4. I tested booting several times – cold boots, restarts and holding the option key to load the Mac boot menu – and the computer always boots normally.

5. FWIW I also then ran `sudo apt full-upgrade` to install the other available updates and the computer had continued to boot normally.

[Balint Reczey (rbalint)]

Verified 1.47+15.4-0ubuntu2 on Hirsute on the MacBook Air (MBA 5,2) I have first observed the problem:

rbalint@chaos:~$ sudo apt install shim-signed
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be upgraded:
  shim-signed
1 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.
Need to get 447 kB of archives.
After this operation, 1.024 B of additional disk space will be used.
Get:1 http://hu.archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 shim-signed amd64 1.47+15.4-0ubuntu2 [447 kB]
Fetched 447 kB in 0s (2.059 kB/s)
Preconfiguring packages ...
(Reading database ... 321160 files and directories currently installed.)
Preparing to unpack .../shim-signed_1.47+15.4-0ubuntu2_amd64.deb ...
Unpacking shim-signed (1.47+15.4-0ubuntu2) over (1.46+15.4-0ubuntu1) ...
Setting up shim-signed (1.47+15.4-0ubuntu2) ...
Installing grub to /boot/efi.
Installing for x86_64-efi platform.
Installation finished. No error reported.
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

rbalint@chaos:~$ Shared connection to chaos-w closed.
$ ssh chaos

rbalint@chaos:~$ sudo ls -alhR /boot/efi/EFI/
/boot/efi/EFI/:
total 16K
drwx------ 4 root root 4,0K jan 18 2018 .
drwx------ 3 root root 4,0K jan 1 1970 ..
drwx------ 2 root root 4,0K febr 4 2019 BOOT
drwx------ 3 root root 4,0K febr 7 2020 ubuntu

/boot/efi/EFI/BOOT:
total 1,9M
drwx------ 2 root root 4,0K febr 4 2019 .
drwx------ 4 root root 4,0K jan 18 2018 ..
-rwx------ 1 root root 933K máj 10 18:24 BOOTX64.EFI
-rwx------ 1 root root 84K máj 10 18:24 fbx64.efi
-rwx------ 1 root root 837K máj 10 18:24 mmx64.efi

/boot/efi/EFI/ubuntu:
total 3,5M
drwx------ 3 root root 4,0K febr 7 2020 .
drwx------ 4 root root 4,0K jan 18 2018 ..
-rwx------ 1 root root 108 máj 10 18:24 BOOTX64.CSV
-rwx------ 1 root root 82K ápr 19 16:33 fbx64.efi
drwx------ 2 root root 4,0K jún 3 2017 fw
-rwx------ 1 root root 126 máj 10 18:24 grub.cfg
-rwx------ 1 root root 1,7M máj 10 18:24 grubx64.efi
-rwx------ 1 root root 837K máj 10 18:24 mmx64.efi
-rwx------ 1 root root 933K máj 10 18:24 shimx64.efi

[MikeMecanic (xyz-t)]

Fresh installed HH final ISO alongside Windows 10 secure boot enabled. Proposed must be enabled in Hirsute (9 packages):

sudo apt install shim-signed
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be upgraded:
  shim-signed
1 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
Need to get 447 kB of archives.
After this operation, 1,024 B of additional disk space will be used.
Get:1 http://mirrors.edge.kernel.org/ubuntu hirsute-proposed/main amd64 shim-signed amd64 1.47+15.4-0ubuntu2 [447 kB]
Fetched 447 kB in 1s (865 kB/s)
Preconfiguring packages ...
(Reading database ... 159131 files and directories currently installed.)
Preparing to unpack .../shim-signed_1.47+15.4-0ubuntu2_amd64.deb ...
Unpacking shim-signed (1.47+15.4-0ubuntu2) over (1.46+15.4-0ubuntu1) ...
Setting up shim-signed (1.47+15.4-0ubuntu2) ...
Trying to migrate /boot/efi into esp config
Installing grub to /boot/efi.
Installing for x86_64-efi platform.
Installation finished. No error reported.

mokutil --sb
SecureBoot enabled
SecureBoot validation is disabled in shim

uname -r
5.13.0-051300rc1-generic

sda 232.9G
├─sda1 vfat 100M /boot/efi
├─sda2 16M
├─sda3 ntfs 150.1G
└─sda4 ext4 82.6G /

All good!

[Launchpad Janitor (janitor)]

This bug was fixed in the package shim - 15.4-0ubuntu2

---------------
shim (15.4-0ubuntu2) hirsute; urgency=medium

  [ Balint Reczey ]
  * Fix boot on EFI 1.10 machines, for example on some MacBooks (LP: #1925010)

  [ Dimitri John Ledkov ]
  * Fix kernel warning when allocating MOK table (LP: #1925139)
  * Fix booting with shim SBState disabled (LP: #1925140)

 -- Dimitri John Ledkov <email address hidden> Tue, 20 Apr 2021 15:24:29 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package shim-signed - 1.47

---------------
shim-signed (1.47) impish; urgency=medium

  [ Balint Reczey ]
  * Fix boot on EFI 1.10 machines, for example on some MacBooks (LP: #1925010)

  [ Dimitri John Ledkov ]
  * Fix kernel warning when allocating MOK table (LP: #1925139)
  * Fix booting with shim SBState disabled (LP: #1925140)
  * Use -Zxz compression, for compatibility with dpkg in older releases.
    LP: #1925673

shim-signed (1.46) hirsute; urgency=medium

  * New upstream release 15.4 LP: #1921134
  * Ship fb & mm from shim-signed package.
  * Remove shim-canonical-unsigned dependency, now provided by shim
    itself.
  * Generalize attaching externally supplied signatures, to aid building
    with embargoed or MS external signatures.

 -- Dimitri John Ledkov <email address hidden> Fri, 30 Apr 2021 10:46:25 +0100

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for shim has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Saptarshi Roy (sroypc)]

Thank you!

[Kris Budde (kebkeb)]

I think the title of this bus is misleading, isn't it?

"shim-signed 15.4 does not boot on EFI 1.10 systems"

shim-signed 15.4 is not affected, it's the fix.

[Steve Langasek (vorlon)]

No. Shim 15.4 upstream is broken. The 15.4-0ubuntu2 includes an additional patch that fixes the problem.

[Łukasz Zemczak (sil2100)]

Hello Balint, or anyone else affected,

Accepted shim into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chris Guiver (guiverc)]

Possibly related - https://bugs.launchpad.net/ubuntu/+bug/1928455 (hirsute, user failing 'fix' causes system to not boot)

[chrisisbd (chris-isbd)]

Should I still be getting "No new release found." when trying to upgrade an xubuntu 20.10 system to 21.04?

It's a bit odd since I have two systems running xubuntu, I upgraded one a few days ago from 20.10 to 21.04 (using do-release-upgrade) but the other system still keeps saying "No new release found." when I run do-release-upgrade.

Did this fix get released and then un-released or something?

[Julian Andres Klode (juliank)]

We are blocked on

https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1928434

and resolving

https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1924605

[pfaelzerchen (pfaelzerchen)]

Sorry, to bother you. I am not sure, if this relates to this bug.

After Upgrades from 20.10 to 21.04 were enabled, I upgraded both of my 20.10 systems to 21.04. On my Intel NUC10 everything worked fine. But my home server were not able to boot anymore. The above-mentioned workaround (replace /EFI/Boot/BOOTX64.efi & /ef/ubuntu/shimx64.efi) helped. The server runs now 21.04.

It is worth noting, that the server was able to boot the Ubuntu 21.04 live USB stick in UEFI mode without trouble.

The board is an older Fujitsu D3417-B with Intel Skylake. Secure boot is disabled.

[Julian Andres Klode (juliank)]

Please open a new bug for that pfaelzerchen, using ubuntu-bug shim on your home server, and provide appropriate logs as documented in

https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1928434

Also check if shim actually loaded grub, and if so, enable grub debug mode, set debug=all
- maybe your issue is related to https://github.com/lxc/lxd/issues/8770

[Julian Andres Klode (juliank)]

Same binaries across all releases, so the verification on hirsute is valid elsewehre too.

[Donald (dmjr21)]

Hi -

I believe I'm experiencing this issue on my Thinkpad X1 Extreme (Gen1). I set up a dual-boot, Ubuntu (Groovy) and Win11 beta. I'd be able to select my OS in a Grub2 menu, after the Lenovo splash screen on start up.

When I was navigating Ubuntu today, I was notified of the available upgrade to 21.04 and went through the steps to upgrade.

At the end of the steps, I needed to reboot my machine:
1) Lenovo splash screen
2) Grub2 menu
3) Select Ubuntu
4) Screen goes black and is unresponsive
5) Hold power to shutdown/restart

After noticing the issue with booting into Ubuntu, I tried to boot into Windows and experienced the same issue - black, unresponsive screen.

I tried several times to boot Ubuntu in recovery mode via advanced options and no dice.

This is a really crummy issue that's rendered my laptop unusable.

-D.

[Julian Andres Klode (juliank)]

Hi, Donald: Your issue is different. Here you can't even get to the grub menu. If you can't boot into Windows either (did you try that from grub, or directly from the EFI boot menu [I think it's F12 or F10 or something you need to press at startup and select Windows]), that sounds more like a firmware issue (if you did not boot via grub).

If you can boot Windows from EFI, but not via grub, and grub and shim are uptodate, please open a new issue. Otherwise, please contact Lenovo for support.

[Donald (dmjr21)]

Hi Julian,

I tried booting Windows via grub and EFI, and Windows tries to repair itself unsuccessfully.

[Donald (dmjr21)]

^ Here are some screenshots.

-D.

[Donald (dmjr21)]

^ Shot showing both failed Ubuntu recovery modes

-D.

[Steve Langasek (vorlon)]

Donald,

I'm very sorry that you're having this experience. It is definitely unrelated to the particular bug this report is about, which, if you were experiencing, would prevent you from ever seeing a GRUB menu.

Have you tried booting the older 5.8.0-59-generic kernel that is still installed on your system? (Not in recovery mode) Perhaps the problem is specific to the new kernel that was installed on your system with 21.04, and booting the older 20.10 kernel will allow you to boot successfully and continue debugging.

Otherwise, I think you will need to avail yourself of one of the support options listed on <https://ubuntu.com/support/community-support>, as this bug report is not the appropriate place to continue debugging your issue.