Unable to boot/install Impish daily in UEFI boot mode

Bug #1937115 reported by Leó Kolbeinsson
60
This bug affects 10 people
Affects Status Importance Assigned to Milestone
OEM Priority Project
Critical
Unassigned
cd-boot-images-amd64 (Ubuntu)
Status tracked in Impish
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Impish
Critical
Unassigned
debian-installer (Ubuntu)
Status tracked in Impish
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Impish
Undecided
Unassigned
shim (Ubuntu)
Status tracked in Impish
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Focal
Undecided
Unassigned
Hirsute
Undecided
Unassigned
Impish
Critical
Unassigned
shim-signed (Ubuntu)
Status tracked in Impish
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Focal
Undecided
Unassigned
Hirsute
Undecided
Unassigned
Impish
Undecided
Unassigned

Bug Description

[Impact]
Removable media fail to boot on some machines due to garbage in firmware generated boot entries. This is a regression from LP: #1929471 introduced in shim 15.4-0ubuntu7.

In addition we also cherry-pick https://github.com/rhboot/shim/pull/365 to fix a potential buffer overflow.

In addition we also cherry-pick https://github.com/rhboot/shim/pull/396 to fix possible firmware corruption in the fallback loader due to out-of-bounds access to the boot order array.

[Test plan]
I'm sure someone can test a new daily impish ISO on an affected machine. Verification is not necessary for individual SRU releases.

We will also do boot of an installed system, and perform the usual chainloaded netbooting test, prior to uploading.

[Regression potential]

We'll apply four patches for the main issue:

Patches 1/2 (https://github.com/rhboot/shim/pull/393) add a fallback to the default loader (with 2s timeout) if we failed to load any specified loader. This ensures we can always boot our default loader, and hence installed systems, even if there is garbage around. It also adds debugging output for the load option that is being parsed, such that people can debug why it failed.

Patches 3 and 4 (https://github.com/rhboot/shim/pull/399) disables parsing load options entirely when booting via the removable media path (boot*.efi). This means that any system where admins generate boot entries like bootx64.efi fwupdx64.efi to load a specific second stage will fail to load that second stage and load grubx64.efi instead.

is this a problem in practice?

On installed systems, we install \EFI\BOOT\bootx64.efi in addition to \EFI\ubuntu\shimx64.efi, and generate boot loader entries for the latter. The former will always use the fbx64.efi fallback loader to add missing boot loader entries and load grub, hence it doesn't seem to support custom options anyway (you'd have to delete fbx64.efi; and even then - you still have the shimx64.efi binary).

On installer media, we do not expect you to specify another second stage loader anyway. It's arguably only problematic there, as those could be read-only and hence you can't rename the binary to shimx64.efi.

For the overflow, the fix is trivially correct.

For the fallback loader, the fix is reasonably trivial; regression potential there could be failure to add a boot entry which is not super critical.

[Original bug report]
Testing Ubuntu Impish daily ISO= http://cdimage.ubuntu.com/daily-live/20210721/impish-desktop-amd64.iso

The test machines are:

1. Dell [ Optiplex] MT 7040 i7-6700 booting in UEFI mode
2. Dell [ Optiplex] MT 7060, i7-8700 booting in UEFI+secure boot mode

Boot from USB media fails with the message:

Failed to open /EFI/boot/? invalid parameter

No further info available

Leó Kolbeinsson (leok)
summary: - Unable to install Impish daily in UEFI boot mode
+ Unable to install Ubuntu Impish daily in UEFI boot mode
Revision history for this message
Ubuntu QA Website (ubuntuqa) wrote : Re: Unable to install Ubuntu Impish daily in UEFI boot mode

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1937115

tags: added: iso-testing
Revision history for this message
Leó Kolbeinsson (leok) wrote : Re: Unable to install Impish daily in UEFI boot mode

This also occurring on Lubuntu Impish daily 2021-07-21
Machine tested Dell [ Optiplex] MT 7060, i7-8700 booting in UEFI+secure boot mode

summary: - Unable to install Ubuntu Impish daily in UEFI boot mode
+ Unable to install Impish daily in UEFI boot mode
Revision history for this message
Leó Kolbeinsson (leok) wrote :

Further tests on 2 Lenovo machines and cannot reproduce the bug.
Strange but seems to only affect Dell machines -- also tested a Dell laptop 7280 aand also could not boot into efi modes.

Revision history for this message
Chris Guiver (guiverc) wrote (last edit ):

Lubuntu impish QA-test install to
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
testcase: full disk, uEFI, ... (fails to boot so doesn't matter)

"INVALID IMAGE"
"FAILED TO BOOT"

then quickly boots to "Attempting to decrypt master key..." of the prior QA-test install located on ssd

the actual messages are more than I typed; but they only flash before switching to the ssd boot which boots the prior QA-test install to the box (an encrypted lubuntu install).

Xubuntu & Ubuntu impish dailies also fail to boot in the same/identical manner. The dailies boot though in other boxes tested.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in syslinux (Ubuntu):
status: New → Confirmed
Chris Guiver (guiverc)
summary: - Unable to install Impish daily in UEFI boot mode
+ Unable to boot/install Impish daily in UEFI boot mode
Chris Guiver (guiverc)
tags: added: amd64 impish
Revision history for this message
Chris Guiver (guiverc) wrote :

Another user has reported issue (yesterday) on askubu

https://askubuntu.com/questions/1353382/any-ubuntu-21-10-iso-download-that-isnt-corrupted

Picture of screen end-user reported can be seen at https://i.stack.imgur.com/oeYYg.jpg

The picture fits what @Leok describes I believe; which is different to the sony vaio I used - however as the sony quickly moved to the next boot device (ssd) it booted a prior installed image meaning I never got a good look at it.

Revision history for this message
tamjk (tamjk) wrote :

Details relating to the machine involved in the previous post by guiverc
========================================================================

Make/Model: Dell Inspiron 5567
CPU: Intel i5-7200U @ 2.5 GHz x 4
RAM: 16GB
GPU: AMD Radeon R7 M445 AMD� Iceland/Mesa; Intel� HD Graphics 620 (KBL GT2)
Desktop Env: Gnome 3.36.8
Windowing System: X11
Storage: 2 x SSD 250 GB + 2 x USB 15/30 GB
Booting: UEFI

tags: added: rls-ii-incoming
1 comments hidden view all 114 comments
Revision history for this message
Bernard Stafford (bernard010) wrote :

Daily 20210721 .iso . This Bug was reported in QA Test Live Session.
Just installed .iso in Virtual Box from a download file,
Tested as Live Session, Success. Loaded good everything worked.
Except for some outdated files that apport would not collect.
Most likely files being used until developed ones finish.
Please check your media and .iso checksum.
Could have a corrupt download or bad USB.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

My media and the .iso checksum are good.

Please note that my success in Virtual Box is due to the fact that it is running in BIOS mode.

This bug has been confirmed by other users as well.

Revision history for this message
Chris Guiver (guiverc) wrote :

Today's boot contains 4 lines (Lubuntu impish) on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

invalid image
failed to load
failed to load
... returns unsupported

the message is still very quick; the screen quickly being cleared and "Attempting to decrypt master key..." from the last successful QA-test install of Lubuntu impish on this box.

The thumb-drive boots normally on other boxes I've tested as usual...

Revision history for this message
Bernard Stafford (bernard010) wrote (last edit ):

Daily 20210721 .iso . This Bug was reported in QA Test Live Session.
Flashed .iso to USB with Etcher. The image failed due to the Linux Header.
Image did not attempt to start, Showed a warning sign.
UEFI boot mode on computer attempting to do live Session.
Lenovo Think Center M78 / 2 core AMD A6-5400B APU / Radeon HD 7540D Graphics
Display: X11
Daily build 20210731 UEFI failed to boot on USB live. Linux Header Failed.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

@bernard010 Bernard Stafford

Please also mark this bug as affecting you and also advise what machine and model this occurred on.

Thank you for testing.

Revision history for this message
Chris Guiver (guiverc) wrote :

Ubuntu impish ISO fails to boot again on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

It fails to boot if ISO is written using mkusb CLONE OR LIVE modes.

(as noted earlier; applies to Xubuntu & Lubuntu too)

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Testing Ubuntu Impish daily ISO 25-07-2021

Tested 8 machines and had 3 boot failures - all machines that failed were Dell machines,and all tests were in UEFI and/or UEFI with secure boot modes.

The boot media was created with "Startup Disk Creator".

The error messages are as follows (see also attached image):

Failed to open \EFI\BOOT\? - Invalid Parameter
Failed to load image \EFI\BOOT\?: Invalid Parameter
start_image() returned Invalid Parameter

Test results posted on the QA testing site -
http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/234198/testcases/1303/results

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Well this is a bit strange. Does anyone know when this started? The way the ISO is put together hasn't changed since the hirsute release, so this is probably related to a change in grub2 or shim. The bug was reported less than 24 hours after a new version of grub2 landed which is definitely something that makes one go "hmm". I'll reassign the package there -- this certainly isn't a syslinux problem as that hasn't been used at all to boot ISOs since focal, and afaik never ever when booting via UEFI.

affects: syslinux (Ubuntu) → grub2 (Ubuntu)
Revision history for this message
Leó Kolbeinsson (leok) wrote :

@mwhudson Michael Hudson Doyle

This bug first appeared with the Ubuntu Impish daily on 21 July 2021 as in the original report -
I test the daily builds every day and can confirm that this was the first instance of the bug.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Seems to be an issue with shim / the UEFI firmware on those machines. Could be a regression from the load option parsing, but confused as to how we end up trying to boot a non existing file instead of falling back.

Could also be memory corruption (shim pull #365).

Need to mokutil --set-verbosity true and record the log during boot with a camera.

affects: grub2 (Ubuntu) → shim (Ubuntu)
Changed in shim (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Julian Andres Klode (juliank) wrote :

Please also include the output of efibootmgr -v on those systems. There is likely garbage in the boot loader entries the UEFI generated for those devices.

Revision history for this message
Leó Kolbeinsson (leok) wrote (last edit ):

@juliank Julian Andres Klode

Sorry for any confusion -- how can I run the commands if I cannot boot the machines?

Revision history for this message
Julian Andres Klode (juliank) wrote :

I have posted a work around for this issue upstream: https://github.com/rhboot/shim/pull/393

This will add a 2s delay on invalid filenames or not found files, and then fallback to the default loader. Hence it's still a good idea to parse the load options properly to avoid the errors in the first place :)

tags: added: regression-proposed
Revision history for this message
Leó Kolbeinsson (leok) wrote (last edit ):

Thanks Julian - will check this out when available.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

The timing of the bug report matches the rebuild of cd-boot-images-amd64 against shim-signed 1.50

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in cd-boot-images-amd64 (Ubuntu):
status: New → Confirmed
Revision history for this message
sudodus (nio-wiklund) wrote :

This bug affects my Dell Latitude E7240 in UEFI mode (but booting works in BIOS mode).

tags: added: fr-1533
Revision history for this message
Julian Andres Klode (juliank) wrote :

I'd still like to see efibootmgr -v output from affected machines, for the boot entry that failed to boot.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Seeing that this affects multiple machines, bumping the priority. Also, looks like we might want to get this fixed before 20.04.3 is released.

Changed in shim (Ubuntu):
importance: Undecided → Critical
Changed in cd-boot-images-amd64 (Ubuntu):
importance: Undecided → Critical
tags: removed: rls-ii-incoming
Changed in shim (Ubuntu Focal):
milestone: none → ubuntu-20.04.3
Revision history for this message
sudodus (nio-wiklund) wrote :

I can report about a workaround:

The current Lubuntu Impish file, impish-desktop-amd64.iso, dated 2021-07-28 17:22 worked for me this way:

- Create a persistent live drive with mkusb-dus.

- Verify that it can boot both in UEFI and legacy BIOS mode.

- Install Lubuntu in UEFI mode (there was no problem for cloned systems to boot in BIOS mode).

But it did not work, when there was a system in the target drive. After wiping the first mibibyte, it was straightforward to install Lubuntu, and I have a working system now :-)

Revision history for this message
sudodus (nio-wiklund) wrote :

I should add, that I installed in a Toshiba,

http://www.toshiba.se/laptops/satellite-pro/c850/satellite-pro-c850-19w/

and then moved the [internal] SSD to a USB3 to SATA adapter and booted my Dell Latitude E7240 and it works in UEFI mode. I don't want to overwrite the internal M2 drive of my Latitude.

Someone else, who has a computer that is affected by this bug may want to try via mkusb-dus and Lubuntu directly in the affected computer. (But it seems to me, that the problem is booting the live drive, not installing and booting after installation.)

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Tested latest Lubuntu Impish ISO dated 29-07-2021 on 2 Dell computers Optiplex 7060 and Optiplex 7040
if I used "startup disk creator" to create the boot media I had 2 failures as in previous tests.

After that I tested the same 2 machines using the suggestion from sudodus(nio-wiklund) and created boot media persistent live media with mkusb-dus - and can confirm that I could boot up in both UEFI and UEFI+secure boot modes.I have not tested booting in BIOS (Legacy) as this has not been a problem.

Hope this is helpful.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Further - here is a link to my test results on the QA site:

http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/234402/testcases/1303/results

Revision history for this message
sudodus (nio-wiklund) wrote :

@leok,

Thanks for testing :-)

I think it could be interesting if you have time to test in one of your Dell Optiplex computers, if it is possible to *install Lubuntu Impish* from a persistent live session, and that the installed system can boot.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

@sudodus (nio-wiklund)

Tested Dell 7060 Optiplex box in UEFI+secure boot mode and installed Lubuntu - rebooted and applied updates and rebooted again - all good -

Revision history for this message
sudodus (nio-wiklund) wrote :

@leok,

Thanks again :-)

Now we can conclude that the booting problem is only affecting the live system, maybe only a cloned live system (we have not yet tested systems made with Rufus; probably such systems will work too).

Revision history for this message
Leó Kolbeinsson (leok) wrote :

@sudodus

I can run a quick check with Rufus created media - will report back later this evening.

Changed in shim (Ubuntu Impish):
status: Incomplete → Triaged
status: Triaged → In Progress
Changed in cd-boot-images-amd64 (Ubuntu Impish):
status: Confirmed → Triaged
Revision history for this message
Julian Andres Klode (juliank) wrote (last edit ):

Hi folks,

we don't need reports of what works and what doesn't.

We already know that only live media are affected - installed systems usually don't boot via removable media path - but if they do, they go via fallback loader binary which works around the issue as it does not interpret its arguments.

We even have two approaches to work around the issue that are under discussion upstream.

Feedback we're looking for for fixing this better would be output of efibootmgr -v with a tarball of /sys/firmware/efi/efivars/Boot* files, on those systems where you can't boot live media. And which of those boot entries did not work.

What happens here is that the bootloader entries in UEFI contain arguments that get passed to shim. Shim tries to parse them, as it wants to treat the first argument as the bootloader to boot next - whcih is used for fwupd. Some firmware simply contains garbage in there, causing things to fail. We'd like to know what this garbage this such that we can safely ignore it and avoid even the workarounds.

Revision history for this message
sudodus (nio-wiklund) wrote :

A USB drive with Impish made by Rufus 'UEFI only' & GPT fails in my Dell Latitude E7240 with the same message as a cloned drive.

@juliank,

Where (in which operating system) and when (at what stage) should we run

efibootmgr -v
tar -cvf boot-debug,tar /sys/firmware/efi/efivars/Boot*

Maybe I'm stupid, but I ask because what we want to run does not boot.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Also tested Rufus "UEFI only" & GPT and fails in Dell Optiplex 7060

I also have no idea (asked about this earlier) as to how to run commands on a machine that does not boot?

What am I missing?

Revision history for this message
Chris Guiver (guiverc) wrote (last edit ):

output of `efibootmgr -v` on the sony crapbook
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
https://paste.ubuntu.com/p/z9rn6Kg625/
FYI: the only OS on it is lubuntu; last focal.3 daily I installed
(these commands were operated from that installed focal.3 system)
---
BootNext: 0005
BootCurrent: 0007
Timeout: 0 seconds
BootOrder: 0005,0006,0007
Boot0005* Sony Original VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
Boot0006* Sony Original VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
Boot0007* Windows Boot Manager HD(1,GPT,6936f223-1e91-eb4e-8498-0ab5af7b9347,0x1000,0x96000)/File(\EFI\Boot\bootx64.efi)
---

sudo tar -zcvf sony_crap.gz /sys/firmware/efi/efivars/Boot*

got the attached results (hopefully no typos)

Revision history for this message
Chris Guiver (guiverc) wrote :

This is another "what works, what doesn't" Sorry (made b/c of focal status at top)

But the
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)
boots the focal daily without issue, even current (20210729).

I've have only had issues with the impish daily.
(the install the prior comment was made from looks like daily 20210716 according to https://phab.lubuntu.me/w/release-team/testing-checklist/ though I'll likely change that shortly)

Changed in cd-boot-images-amd64 (Ubuntu Focal):
status: New → Confirmed
Changed in shim (Ubuntu Focal):
status: New → Confirmed
description: updated
tags: added: regression-update
description: updated
description: updated
description: updated
description: updated
tags: added: oem-priority
Changed in oem-priority:
importance: Undecided → Critical
status: New → Confirmed
Changed in shim (Ubuntu Focal):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-focal
Changed in shim (Ubuntu Impish):
status: In Progress → Fix Released
Changed in shim-signed (Ubuntu Focal):
status: New → Fix Committed
Changed in shim-signed (Ubuntu Impish):
status: New → Fix Released
no longer affects: cd-boot-images-amd64 (Ubuntu Focal)
Changed in cd-boot-images-amd64 (Ubuntu Impish):
status: Triaged → Fix Committed
Changed in cd-boot-images-amd64 (Ubuntu Impish):
status: Fix Committed → Fix Released
34 comments hidden view all 114 comments
Revision history for this message
Julian Andres Klode (juliank) wrote :

Apparently shim is not pre-installed in focal, but installed from pool, so yes, file list is correct:

/pool/main/s/shim-signed/shim-signed_1.40.7+15.4-0ubuntu9_amd64.deb

So this is good, marking as verified then

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
sudodus (nio-wiklund) wrote :

Booted the current Ubuntu iso files from the iso testing web-site

-rw------- 2,9G 2021-08-16 08:21 "focal-desktop-amd64.iso"
-rw------- 2,9G 2021-08-16 08:30 "impish-desktop-amd64.iso"

in a Dell Latitude E7240 in UEFI mode.

The impish iso file fails :-(

Then I read the latest posts by you developers, so I zsynced the focal iso file and it boots :-)

Revision history for this message
Bernard Stafford (bernard010) wrote :

Daily build 20210816.iso REJECT The Header, Kernel, Modules & Extra Modules UNSUPPORTED! For UEFI Install using USB.. Latest FIX is a FAILURE. COMPLETE FAIL< WILL NOT BOOT...
Latest Kernel I have tested that works is 5.13.11 @ kernel.org. Works Awesome for this Version of Ubuntu impish.
Tested 4 times.

Revision history for this message
Bernard Stafford (bernard010) wrote (last edit ):

KERNEL FAILED ON UEFI BOOT Not Fixed...!!!
UPDATE KERNEL Please!
Daily build 20210816.iso REJECT The Header, Kernel, Modules & Extra Modules UNSUPPORTED! For UEFI Install using USB.. Latest FIX is a FAILURE. COMPLETE FAIL< WILL NOT BOOT...
Latest Kernel I have tested that works is 5.10.60 @ kernel.org. Works Awesome for this Version of Ubuntu Impish.
Tested 4 times. With new current Kernel

Revision history for this message
Chris Guiver (guiverc) wrote :

Both Lubuntu & Ubuntu(desktop) focal dailies (2021-08-16) boot on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

(the only box I have that was impacted by this bug)

Revision history for this message
Brian Murray (brian-murray) wrote :

The daily images from 20210817 should have the latest version of shim on them now so feel free to test that.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Tested Kubuntu daily ISO 170821 no boot errors

see: http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235402/testcases/1300/results

for details.

Revision history for this message
sudodus (nio-wiklund) wrote :

Tested the current Lubuntu Impish iso file dated

2021-08-17 17:22 "impish-desktop-amd64.iso"

Booting a cloned USB drive works [not only in BIOS mode but also] in UEFI mode in a Dell Latitude E7240 :-)

So I can confirm that the bugfix works also to boot Lubuntu Impish in my Dell.

Revision history for this message
sudodus (nio-wiklund) wrote :

I tested the current Lubuntu Impish live also in a Dell Precision M4800. It works in UEFI mode too.

Revision history for this message
Leó Kolbeinsson (leok) wrote :

Another confirm that bux fix good in Lubuntu

Tested Lubuntu Impish daily ISO 17-08-2021

Results here = http://iso.qa.ubuntu.com/qatracker/milestones/424/builds/235442/testcases/1701/results

InWin BL641 BIOS mode in VirtBox - success
Dell optiplex 7040 in UEFI mode - success
Dell optiplex 7060 in UEFI+ secure boot mode - success

Revision history for this message
Steve Langasek (vorlon) wrote :

Thanks, this is a generic fix that has been confirmed, verified for focal and released for impish. No further tests/reports needed.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Thank you for all the verification! I also confirmed this shim is working correctly on physical hardware - let me release it into focal-updates in a few moments.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

Changed in shim (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim-signed - 1.40.7

---------------
shim-signed (1.40.7) focal; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 13 Aug 2021 18:07:24 +0200

Changed in shim-signed (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for shim has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Bernard Stafford (bernard010) wrote :

QA Testing of Live daily build 20210819. UEFI boots properly flashed to USB key with Etcher.
This Bug Report is sufficiently satisfied.

Revision history for this message
sudodus (nio-wiklund) wrote :

@ Julian Andres Klode (juliank),

Please explain:

How does this bug affect the old versions Xenial and Bionic?

And how should we test that our computers are affected?

Revision history for this message
Bob H (bobbicat) wrote (last edit ):

recently... while booting the ISO image for the Impish live build
the following error lines are displayed:

[ 1.501001] gpio gpiochip: (gpio_aaeon): tried to insert a GPIO chip with zero lines
[ 1.501013] gpiochip_add_data_with_key: GPIOs O..-1 (gpio_aaeon) failed to register. -22
[ 0.197133] gpio gpiochip: (gpio_aaeon): tried to insert a GPIO chip with zero lines
[ 0.197161] gpiochip_add_data_with_key: GPIOs O..-1 (gpio_aaeon) failed to register. -22

After a pause the iso then continues to boot and a successful installation proceeds without any signs of further error.

this first occurred at 20210820 (since fix released)

I have an own build PC with nVidia graphics and use 'safe graphics' to boot the iso
Prior to this I was not experiencing the problems that had been occurring with Dell laptops.

Revision history for this message
Bob H (bobbicat) wrote :

The matter reported in post #92 appears to be in process of being dealt with in Bug#1937897

Changed in oem-priority:
status: Confirmed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Leó, or anyone else affected,

Accepted shim into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in shim (Ubuntu Hirsute):
status: New → Fix Committed
tags: added: verification-needed verification-needed-hirsute
removed: verification-done
Changed in shim-signed (Ubuntu Hirsute):
status: New → Fix Committed
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Leó, or anyone else affected,

Accepted shim-signed into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.51 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Leó, or anyone else affected,

Accepted shim into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in shim (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Leó, or anyone else affected,

Accepted shim-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.37~18.04.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in shim-signed (Ubuntu Bionic):
status: New → Fix Committed
Revision history for this message
Julian Andres Klode (juliank) wrote :

Marking as verified as per

"Verification is not necessary for individual SRU releases."

We did not touch any code paths involving shim-grub interaction, the fix here for disabling boot option parsing on removable media paths is valid across all releases, since the binary is the same.

tags: added: verification-done verification-done-bionic verification-done-hirsute
removed: verification-needed verification-needed-bionic verification-needed-hirsute
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

Changed in shim (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim-signed - 1.51

---------------
shim-signed (1.51) impish; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 13 Aug 2021 18:00:15 +0200

Changed in shim-signed (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim-signed - 1.37~18.04.11

---------------
shim-signed (1.37~18.04.11) bionic; urgency=medium

  * Update to shim 15.4-0ubuntu9
    - Fix booting installer media on some machines (LP: #1937115)
      + Always fallback to the default loader (PR #393)
      + Dump load options parsed (PR #393)
      + Disable load option parsing on removable media path (PR #399)
    - trivial: Fix a minor overflow in the mok importing code (PR #365)
    - Fix fall back loader to find the correct boot entry, avoiding potential
      corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Tue, 07 Sep 2021 12:03:11 +0200

Changed in shim-signed (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package shim - 15.4-0ubuntu9

---------------
shim (15.4-0ubuntu9) hirsute; urgency=medium

  * Fix booting installer media on some machines (LP: #1937115)
    - Always fallback to the default loader (PR #393)
    - Dump load options parsed (PR #393)
    - Disable load option parsing on removable media path (PR #399)
  * trivial: Fix a minor overflow in the mok importing code (PR #365)
  * Fix fall back loader to find the correct boot entry, avoiding potential
    corruption of firmware (PR #396).

 -- Julian Andres Klode <email address hidden> Fri, 06 Aug 2021 13:16:33 +0200

Changed in shim (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Paride Legovini (paride) wrote :

This has been reported in the ISO tracker as affecting the 18.04.6 candidate images (20210913):

http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236713/testcases

The reported issue happens on bare metal. In QEMU/KVM the 20210913 bionic-live-server-amd64 images work fine with secureboot enabled (checked via `mokutil --sb-state` in both the installer and installed systems).

Revision history for this message
Leó Kolbeinsson (leok) wrote :

This has also been reported in the ISO tracker as affecting the 18.04.6 candidate images Ubuntu Desktop (20210913):

http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236704/testcases/1300/results

These tests on bare metal machines as were the server tests in comment # 103 above.

Revision history for this message
Steve Langasek (vorlon) wrote :

This is because debian-installer somehow was uploaded for 18.04.6 built against shim-signed 1.37~18.04.10+15.4-0ubuntu7 instead of shim-signed 1.37~18.04.11+15.4-0ubuntu9. Not sure how or why this happened, but shim-signed has been released to bionic-updates now, so I'll rebuild debian-installer for the fix.

Changed in cd-boot-images-amd64 (Ubuntu Xenial):
status: New → Invalid
Changed in cd-boot-images-amd64 (Ubuntu Bionic):
status: New → Invalid
Changed in debian-installer (Ubuntu Impish):
status: New → Invalid
Changed in debian-installer (Ubuntu Xenial):
status: New → Invalid
Changed in debian-installer (Ubuntu Bionic):
status: New → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Leó, or anyone else affected,

Accepted debian-installer into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debian-installer/20101020ubuntu543.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in debian-installer (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
removed: verification-done verification-done-bionic
Revision history for this message
Chris Guiver (guiverc) wrote (last edit ):

failure to boot 18.04.6 ISO (20210913.1) on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

Can't add -proposed (easily) to a system that fails to boot :(

4 error lines briefly appear on screen before box moves to next boot device; which is SSD & lubuntu impish GPG key is requested..

Revision history for this message
Paride Legovini (paride) wrote (last edit ):

@Steve I'm still a bit puzzled by the reported failure [1], which is against the live-server (subiquity) image, and not against the d-i image, but according to the problem description the failure mode is similar (image fails to boot).

Could it be that some other component in the live-server image was built against the old shim, as it happened for debian-installer?

[1] http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236713/testcases

Revision history for this message
Steve Langasek (vorlon) wrote :

The boot bits of all the hybrid images are laid down by debian-cd, which for bionic takes those bits from the debian-installer build, and not from the archive directly.

Revision history for this message
Paride Legovini (paride) wrote :

That's reassuring, thanks Steve. Now waiting for a respin to proceed with the tests.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

@Paride So I don't know these 'legacy' boot bits, but quoting Steve from yesterday:

"vorlon: yes, the boot bits for both images are copied from debian-installer output. In later releases this is done via xnox's cd-boot-images instead"

So possibly this might also cause issues for all isos?

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package debian-installer - 20101020ubuntu543.19

---------------
debian-installer (20101020ubuntu543.19) bionic; urgency=medium

  * Fix Vcs-Bzr branch target.
  * Rebuild against shim-signed 1.37~18.04.11+15.4-0ubuntu9. LP: #1937115.

 -- Steve Langasek <email address hidden> Tue, 14 Sep 2021 14:36:54 -0700

Changed in debian-installer (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Chris Guiver (guiverc) wrote :

Ubuntu 18.04.6 Desktop ISO (20210915) now boots on
- sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

(failure in comment #107)

Revision history for this message
Leó Kolbeinsson (leok) wrote (last edit ):

I can also confirm that Ubuntu 18.04.06 desktop and server ISO's (29210915) boot on boxes that previously failed - see my comments # 103 and 104 above.

Test results on QA tracking site here:

1. Desktop tests = http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236832/testcases/1303/results

2. Server tests = http://iso.qa.ubuntu.com/qatracker/milestones/426/builds/236831/testcases/1697/results/

Displaying first 40 and last 40 comments. View all 114 comments or add a comment.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.