Can't boot: "error: out of memory." immediately after the grub menu

Bug #1842320 reported by Gordon Mckeown
342
This bug affects 63 people
Affects Status Importance Assigned to Milestone
OEM Priority Project
Triaged
Critical
jeremyszu
grub
Unknown
Unknown
grub2-signed (Ubuntu)
Fix Committed
Critical
Julian Andres Klode
Bionic
In Progress
Undecided
Unassigned
Focal
In Progress
Undecided
Unassigned
Jammy
In Progress
Undecided
Unassigned
Kinetic
In Progress
Undecided
Unassigned
grub2-unsigned (Ubuntu)
Fix Released
Critical
Julian Andres Klode
Bionic
In Progress
Undecided
Unassigned
Focal
In Progress
Undecided
Unassigned
Jammy
In Progress
Undecided
Unassigned
Kinetic
In Progress
Undecided
Unassigned

Bug Description

[Workaround]

Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125

[Impact]

 * In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.

 * Some real cases from OEM projects:

In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.

In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:

```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
    at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
    type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```

Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:

Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
  LoaderCode: 562 Pages (2,301,952 Bytes)
  LoaderData: 0 Pages (0 Bytes)
...
  Available : 1,917,598 Pages (7,854,481,408 Bytes)

Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().

As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.

 * When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).

[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.

 * detailed instructions how to reproduce the bug:

1. Any method to grow the initramfs, such as install nvidia-driver.

2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:

```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh

PREREQ=""

prereqs()
{
        echo "$PREREQ"
}

case $1 in
# get pre-requisites
prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```

And then update-initramfs

 * After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:

421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail

Only 453M is because verifier will consume the same memory size of initrd in <1G memories.

The loadable initrd will locate at >4G if machine/kernel support it.

[Where problems could occur]

* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.

* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.

If everything works as expected, then i386 should working good.

If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.

[Other Info]

 * The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.

---

Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.

On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.

Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):

              total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516

Kernel packages installed:

linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64

Photo of kernel panic attached.

NVMe drive partition layout (GPT):

Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem

$ sudo pvs
  PV VG Fmt Attr PSize PFree
  /dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0

$ sudo lvs
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  root ubuntu-vg -wi-ao---- 474.75g
  swap_1 ubuntu-vg -wi-ao---- 980.00m

Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 001 Device 003: ID 8087:0a2b Intel Corp.
 Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
 Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
 linux-restricted-modules-5.0.0-25-generic N/A
 linux-backports-modules-5.0.0-25-generic N/A
 linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP

Revision history for this message
Gordon Mckeown (thefluffyone) wrote :
description: updated
Paul White (paulw2u)
affects: ubuntu → linux (Ubuntu)
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1842320

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
tags: added: disco
Revision history for this message
Gordon Mckeown (thefluffyone) wrote : AlsaInfo.txt

apport information

tags: added: apport-collected eoan
description: updated
Revision history for this message
Gordon Mckeown (thefluffyone) wrote : CRDA.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : IwConfig.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : Lspci.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : ProcEnviron.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : ProcModules.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : PulseList.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : RfKill.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : UdevDb.txt

apport information

Revision history for this message
Gordon Mckeown (thefluffyone) wrote : WifiSyslog.txt

apport information

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Gordon Mckeown (thefluffyone) wrote : Re: Out of Memory on boot with 5.2.0 kernel

Issue appears to be related to display resolution. Grub was booting into 3840x2160 mode, with the resulting tiny, near-unreadable font. So I set GFXMODE=800x600 and now - unexpectedly - kernel 5.2.0 is able to boot without the memory errors.

Not sure why the high resolution causes an out-of-memory error; a 3840x2160x32 framebuffer should only require ~32MB of RAM.

Revision history for this message
MrMEEE (mj-casalogic) wrote :

Same issue here.. workaround works..

Revision history for this message
Kai-Heng Feng (kaihengfeng) wrote :

Can you please perform a kernel bisection?

Revision history for this message
Steven Mackenzie (z-steven) wrote :

I have a similar issue. I upgraded to 19.10 from 19.04 today.

After grub, I get this message:

"error: out of memory.
Press any key to continue..."

Then kernel panic message (image attached).

I have been unable to successfully boot using the workaround above.

The kernel is shown in grub is 5.3.0-18.
My computer is an Intel NUC (Skull Canyon), 16GB RAM, Intel NVMe, 2 x 2560 x 1440 displays.

Revision history for this message
Steven Mackenzie (z-steven) wrote :

Update: My machine boots normally from the USB install media.

Revision history for this message
Terry Dawson (tjd-animats) wrote : apport information

ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu8.2
Architecture: amd64
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
DistroRelease: Ubuntu 19.10
HibernationDevice: RESUME=UUID=21a8b99d-8777-4e3e-8fd7-2ad7bb66b24e
InstallationDate: Installed on 2017-01-24 (1068 days ago)
InstallationMedia: Ubuntu 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
IwConfig:
 lo no wireless extensions.

 eno1 no wireless extensions.
Package: linux (not installed)
ProcEnviron:
 LANGUAGE=en_AU:en
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=en_AU.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.4.0-131-generic root=/dev/mapper/ubuntu--vg-root ro plymouth:debug
ProcVersionSignature: Ubuntu 4.4.0-131.157-generic 4.4.134
PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No PulseAudio daemon running, or not running as session daemon.
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-131-generic N/A
 linux-backports-modules-4.4.0-131-generic N/A
 linux-firmware 1.183.3
RfKill:

Tags: eoan
Uname: Linux 4.4.0-131-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-12-28 (0 days ago)
UserGroups:

_MarkForUpload: True
dmi.bios.date: 08/17/2016
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: KYSKLi70.86A.0041.2016.0817.1130
dmi.board.name: NUC6i7KYB
dmi.board.vendor: Intel Corporation
dmi.board.version: H90766-406
dmi.chassis.type: 3
dmi.chassis.vendor: Intel Corporation
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrKYSKLi70.86A.0041.2016.0817.1130:bd08/17/2016:svn:pn:pvr:rvnIntelCorporation:rnNUC6i7KYB:rvrH90766-406:cvnIntelCorporation:ct3:cvr1.0:

Revision history for this message
Terry Dawson (tjd-animats) wrote : AlsaInfo.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : Lspci.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : Lsusb.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : ProcModules.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : UdevDb.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : WifiSyslog.txt

apport information

Revision history for this message
Terry Dawson (tjd-animats) wrote : Re: Out of Memory on boot with 5.2.0 kernel

I'd like to echo Steven Mackenzie's report.

The workaround of explicitly configuring GFXMODE does mean that I can hit a key through the "Hit any key to continue" prompt, but the boot still fails. Without the workaround, grub is unresponsive at the "Hit any key.." prompt.

Revision history for this message
Terry Dawson (tjd-animats) wrote :

I've opened 1857786 indepdently for my boot issue.

Revision history for this message
Ben Hoyt (benhoyt) wrote :

Just noting that I had this same issue on my Dell XPS 9550 laptop, which has a 3840x2160 display. The workaround of setting GFXMODE=800x600 in the grub config worked for me too, thanks! And of course had the nice side effect of actually making the grub menu readable. :-)

Revision history for this message
Gordon Mckeown (thefluffyone) wrote :

Just had the problem again after upgrading 20.10 to 21.04.

Kernel 5.8.0-55 boots OK.

Kernel 5.11.0-18 fails with the memory error.

I'm just about to re-add the GFXMODE line to grub, which I hope will (again) work around the problem.

Revision history for this message
H (bluesforte) wrote :

I had this issue as well, through ALL of the 5.x kernels. Tried everything mentioned in the thread here, but nothing worked other than going to 4.15 kernel.
This gave me suspicions it was related to something with the UEFI handoff to OS, so I looked for BIOS updates and sure enough there was one from earlier this year. Updated to the latest BIOS from Intel, and everything boots fine and working now.

Revision history for this message
Callum L (little-king-john) wrote :

I have this issue on my Dell XPS 9380. I can boot 5.13.0.35, but 0.37 and 0.39 simply show the error: out of memory.

I have tried changing my GFXMODE to 800x600x24, which hwinfo lists as a supported resolution, but grub.... just ignores it? Same with timeout? It just shows in the 4K native res every time, almost as if it's taunting me.

Eventually the 0.35 kernel will be deleted by a system update, and my system will be bricked. I guess I'm going to need to reinstall, aren't I?

Revision history for this message
Paulo (ptsneves) wrote :

This is still valid and the issue is more pernicious because with the latest livecd one cannot change the GFXMODE to 800x600. Re-imaging a new ISO with new grub settings is outside most people's abilities nor is it the intention. This means that Ubuntu's livecd lead to a panic in a very popular laptop model.

Revision history for this message
rustyx (rustyx2) wrote :

Had the same issue with Ubuntu 22.04 install ISO flashed onto a USB.

Even tried booting "manually" (press "c" to enter GRUB command prompt)

grub> ls
(proc) (hd0) (hd0,gpt1) (hd0,gpt2) (hd0,gpt3) (hd1) . . .
grub> ls (hd0,gpt1)
        Partition hd0,gpt1: Filesystem type ... - Label 'UBUNTU 22_0' ...
grub> set root=(hd0,gpt1)
grub> linux /casper/vmlinuz
grub> initrd /casper/initrd
error: out of memory.
grub>

Couldn't find any way out of this condition (is there?).

Fortunately I already had GRUB installed on the machine so I could proceed from that one.

Chris Guiver (guiverc)
tags: added: jammy
removed: disco eoan
jeremyszu (os369510)
tags: added: jiayi oem-priority originate-from-1972964
jeremyszu (os369510)
Changed in oem-priority:
assignee: nobody → jeremyszu (os369510)
importance: Undecided → Critical
status: New → In Progress
jeremyszu (os369510)
tags: added: jellyfish-edge-staging
Changed in grub2-signed (Ubuntu):
status: New → Confirmed
Changed in initramfs-tools (Ubuntu):
status: New → Confirmed
jeremyszu (os369510)
description: updated
description: updated
Changed in oem-priority:
status: In Progress → Triaged
tags: added: patch
jeremyszu (os369510)
Changed in oem-priority:
status: Triaged → In Progress
jeremyszu (os369510)
Changed in oem-priority:
status: In Progress → Triaged
summary: - Out of Memory on boot with 5.2.0 kernel
+ Out of Memory on boot
tags: added: kinetic
Changed in grub2-signed (Ubuntu):
importance: Undecided → Critical
Changed in initramfs-tools (Ubuntu):
importance: Undecided → Critical
Changed in linux (Ubuntu):
importance: Undecided → Critical
summary: - Out of Memory on boot
+ Can't boot: "error: out of memory." immediately after the grub menu
tags: added: foundations-todo
Steve Langasek (vorlon)
Changed in grub2-signed (Ubuntu):
status: Confirmed → Triaged
Changed in initramfs-tools (Ubuntu):
status: Confirmed → Won't Fix
66 comments hidden view all 146 comments
Revision history for this message
Peter-and-Crazybear (crazybear) wrote : Re: [Bug 1842320] Re: Can't boot: "error: out of memory." immediately after the grub menu

When is this going to be fixed and is there a work-around. I have tried
both updating from 20.04 and direct installation of 22.04 and have found
this problem preventing me from using 22.04 [it never boots]. I have two
computers. One is a Dell with nvidia drivers [the perfect target machine
to show this problem] - and it fails, but I also have a desktop I built
myself that should not have problems [and obviously some having
installed 22.04 do NOT have problems, but I do - and can not tell you
what component is the problem [hardware or software]. Somewhere I read
that making some spaces larger for the possible 'expansion' to not cause
problems, but can not find it now with all of the many entries. I have
spent MONTHS trying to install 22.04....without success on either
machine. I do not think you are correct that it only effects a small
number of machines. Anyway, for those of us who it does, the problem is
IMMENSE and total.

On 03/08/2022 10:44, Julian Andres Klode wrote:
> Yes, grub will be fixed eventually, but we are blocked by the security
> update not being out yet. This is not a blocker for enabling upgrades to
> 22.04.1, as it only affects a small number of systems and grub fixes
> itself by using the previous kernel version if the new one fails.
>

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Workaround:

You can usually install Ubuntu on a drive in one machine and then move the drive into another machine.

At least I think that's what I did on the affected machine with 22.04, which is why I never discovered this bug until 22.10 development started.

Revision history for this message
sudodus (nio-wiklund) wrote :

Another workaround might be to use Ubuntu's own compressed image with a preinstalled Ubuntu Server (although not officially released) according to the following link

https://ubuntuforums.org/showthread.php?t=2474692

Scroll down to read also the following posts, that describe how to install Lubuntu. In the same way you can install Ubuntu Desktop as well as all the community flavours.

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Duplicate bug 1970402 mentions that:

> Workaround: Disable Secure Boot and SGX.

works for multiple people.

Revision history for this message
Stefan Akatyschew (fried-gluttony) wrote :

I can replicate this on my Dell XPS13 9380 (no encryption). Occurs as well with Ubuntu 20.04 LTS, but not with 18.04 LTS, so not sure if it is related to the same reason?
-> Disabling Secure Boot and SGX did not resolve the issue

Revision history for this message
Julian Andres Klode (juliank) wrote :

Really the workaround is to set MODULES=dep in /etc/initramfs-tools/initramfs.conf, this yields a much smaller initramfs (but you can't take out the disk and boot it in another machine), disabling secure boot or sgx should not be doing anything.

Changed in grub2-unsigned (Ubuntu):
status: New → Triaged
importance: Undecided → Critical
Revision history for this message
Adrien Nader (adrien-n) wrote :

As I've said elsewhere, if we dedup firmware files through symlinks, we can save 10MB in initrds. Compression does not help because the compressors have very small compression windows and cannot see redundancy in practice (this applies to xz to a lesser extent but even for xz there is an improvement to be gained).

I don't know if it will be enough but it could tip us just below some threshold and it's easy to implement. It should also save memory at runtime.

And for reference, there's a working but weird work-around that only touches initramfs.conf, keeps MODULES=most and is reasonably fast. Use 'xz' as the compress program and then add the following on a new line:

  export XZ_OPT=--lzma2=preset=0,dict=16M

(yes, it relies on implementation details of mkinitramfs)

Revision history for this message
ybdjkfd (ybdjkfd) wrote :

I upgraded from 22.04.1 to 22.10. Issue returned as /usr/sbin/mkinitramfs was changed to default. Had to follow my previous steps shown after #41 . . . zstd had defaulted to a lower compression value in /usr/sbin/mkinitramfs at 1 instead of 19. I changed line 196 to 19 compression

    zstd) compress="zstd -q -19 -T0" ;;

Instead of

    zstd) compress="zstd -q -1 -T0" ;;

Problem solved from proper chroot and updating grub2 after changing line 196 of /usr/sbin/mkinitramfs for my system.

I can boot normally again and any upgrades to initramfs don't cause this failure.

Revision history for this message
Benjamin Drung (bdrung) wrote :

initramfs-tools 0.142 added support for specifying the compression level. Ubuntu will probably merge this version for Ubuntu 23.04.

Andy Chi (andch)
tags: added: originate-from-1994098 stella
Revision history for this message
jeremyszu (os369510) wrote (last edit ):

If possible, I still expect we can make grub supports to boot from bigger initramfs.

There will still have users to meet this issue in the future if their initramfs somehow bigger as long as we don't choose use higher compression as workaround (for ubuntu desktop / server).

Revision history for this message
Andy Chi (andch) wrote :

Download ubuntu-20.04.5-desktop-amd64.iso and try to install on a HP laptop, I can't boot into installation menu. If initrd.img lower than 100MB then everything is fine.

Revision history for this message
iMac (imac-netstatz) wrote (last edit ):

My XPS 13 9380 (no luks) also triggers this on 5.19 after the 22.10 upgrade; I can see the initrd is about 7% bigger in 22.10 over 22.04.

 -rw-r--r-- 1 root root 111203941 Oct 30 18:16 initrd.img-5.15.0-52-generic
 -rw-r--r-- 1 root root 120222346 Oct 31 21:39 initrd.img-5.19.0-23-generic

#103 lays out some interesting blockers for grub2, so as of right now there is no simple way to get upstream through ubuntu repositories as grub2 2.06-2ubuntu12 does not have a fix.

I think it is interesting that in https://lists.gnu.org/archive/html/grub-devel/2017-03/msg00037.html there was pushback on autodetection to fix this, citing "that machine that nobody heard of that will fail" however it ended up being a bunch of not-yet-built systems that failed

Looking at initramfs-tools in Jammy, the very last changelog entry was the change to lower compression, that is triggering this issue:

 http://changelogs.ubuntu.com/changelogs/pool/main/i/initramfs-tools/initramfs-tools_0.140ubuntu13/changelog

 initramfs-tools (0.140ubuntu13) jammy; urgency=medium

  * Lower the compression levels for zstd and lz4 (LP: #1958148)
    Following the discussion on the mailing list, we have reached
    a conclusion to lower the default compression levels:
    - For lz4, the compression level is lowered to 2 from 9
    - For zstd, the compression level is lowered to 1 from 19

Simple fix IMHO is to just go back one version. Ubuntu main archive doesn't include all old versions, but other mirrors do. Here are some quick steps that let you run official binaries from Jammy under Kinetic to work around this issue until grub2 catches up and/or they tweak compression, or make compression more easily tunable.

wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools-bin_0.140ubuntu12_amd64.deb
wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools-core_0.140ubuntu12_all.deb
wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools_0.140ubuntu12_all.deb
dpkg -i initramfs-tools-bin_0.140ubuntu12_amd64.deb
dpkg -i initramfs-tools-core_0.140ubuntu12_all.deb
dpkg -i initramfs-tools_0.140ubuntu12_all.deb
apt-mark hold initramfs-tools-bin
apt-mark hold initramfs-tools-core
apt-mark hold initramfs-tools
update-initramfs -u

Afterwards, big drop in initrd size. 34% smaller. I am definately getting those fractions of a second back on NVME boot.
-rw-r--r-- 1 root root 111203941 Oct 30 18:16 initrd.img-5.15.0-52-generic
-rw-r--r-- 1 root root 79034965 Oct 31 22:50 initrd.img-5.19.0-23-generic

Sometime down the road, when this bug is closed, you will want to un-hold these to restore the lastest packages

 apt-mark unhold initramfs-tools-bin
 apt-mark unhold initramfs-tools-core
 apt-mark unhold initramfs-tools

Revision history for this message
jeremyszu (os369510) wrote :

I don't think "MODULES=dep" is the proper workaround for this.

If an user somehow met this issue, then the system is not able to boot.
Users need to do many extra works to find out this workaround and do apply unless we make "MODULES=dep" as default setting in initramfs configs (but I don't think it's doable).

Based on #117, we also confirm the stock ubuntu not able to boot on some machines.
The fix needs to be included in the packages from ubuntu-archive as common solution.

Julian,

Can we go back to consider #105, #106 to try to land it in Lunar or Kinetic first?

Revision history for this message
iMac (imac-netstatz) wrote :

-> If an user somehow met this issue, then the system is not able to boot.

It can boot, if it did previously, and this issue will get fixed pretty quick IMHO in the updates repository given it impacts current 20.04, 22.04 and 22.10.

Here is just the workaround part, for current 22.10 or 22.04 to help people stuck on boot that used to have a working system.

1) Hold the SHIFT key on boot, and select the 2nd newest kernel from the advanced list in the grub menu: i.e. for myself, 5.19 broke, so using the older 5.15 allowed me to boot.

2) Open a terminal, sudo -i to root, and run these commands to download, install, and prevent upgrades to last version of initramfs-tools before the compression rates were changed, which allows you to regenerate your initrd image with the previously working smaller size.

Download:
 wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools-bin_0.140ubuntu12_amd64.deb
 wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools-core_0.140ubuntu12_all.deb
 wget https://ubuntu.repo.cure.edu.uy/mirror/pool/main/i/initramfs-tools/initramfs-tools_0.140ubuntu12_all.deb

Install:
 dpkg -i initramfs-tools-bin_0.140ubuntu12_amd64.deb
 dpkg -i initramfs-tools-core_0.140ubuntu12_all.deb
 dpkg -i initramfs-tools_0.140ubuntu12_all.deb

Hold Upgrades:
 apt-mark hold initramfs-tools-bin
 apt-mark hold initramfs-tools-core
 apt-mark hold initramfs-tools

Re-generate current initrd image:
 update-initramfs -u

A reboot should just bring you up in 5.19. To restore the system to a normal state once the problem has been fixed in the current versions, unhold those same packages using the steps below, and the new ones will automatically be downloaded and installed.

 apt-mark unhold initramfs-tools-bin
 apt-mark unhold initramfs-tools-core
 apt-mark unhold initramfs-tools

Some additional notes, not for the workaround above follow.

Do NOT issue the "update-initramfs -k all" command, if you have been thinking about it as a first triage step. The reason being, if you have not applied some kind of fix or workaround prior, this will rebuild all your old previously working kernel initrd images with the new low compression directive, which might render them all unbootable, further complicating matters. If you get to this state, and you need a working live ISO, to chroot and fix, make sure to choose one prior with the older, increased initrd compresssion which must be 20.04.4 LTS or older. Current 20.05.5, and all 22.04 and 22.10 releases use the lower compression that can cause this issue and may not work. Impatient new installers may find this info useful as well.

Another simpler workaround for more advanced users with 4K screens is to pass GFXMODE=640x480 in grub defaults and update-grub as mentioned here: http://savannah.gnu.org/bugs/?61058

tags: added: fr-2934
Changed in grub2-unsigned (Ubuntu):
assignee: nobody → Julian Andres Klode (juliank)
Changed in grub2-signed (Ubuntu):
assignee: nobody → Julian Andres Klode (juliank)
Revision history for this message
Yohn Chrichton (yokrin) wrote :

The issue happened for me after installing newer nvidia proprietary drivers (515 and also 520).
The only way to recover was to boot using a previous installed kernel (I used rescue mode. The rescue mode in current kernel still gives the same error)

Not sure how this helps this bug investigation but was real pain knowing that I couldn't install the nvidia drivers.

The only thing that worked was installing the nvidia drivers 470 version.

https://forums.developer.nvidia.com/t/ubuntu-22-04-1-nvidia-driver-open-kernel-nvidia-driver-515-open-issue/231356/12?u=yohn.krichton

Revision history for this message
Craig Hesling (hesling-craig) wrote (last edit ):

Confirmed that reducing the size of the initrd.img slightly allowed my machine to boot. I just enabled `COMPRESSLEVEL=19` with `COMPRESS=zstd` (default on debian) in initramfs.conf. That reduced my initrd.img size from 72MB to 62MB, which allowed my machine to boot.

Ultimately, I set `MODULES=dep` (instead of `MODULES=most`) and reverted the compression to defaults. This reduced my initrd.img to 22MB, which includes Intel microcode (specific binary), nvme, crypto, and lvm support, among other things.

---

For the record:

My machine is running Debian bookworm with the Nvidia proprietary drivers with 32GB of RAM. I first noticed the issue when, coincidentally, upgrading from kernel 5.19 to 6.0.

```
Loading Linux 6.0.0-4amd64 ...
Loading initial ramdisk ...
error: out of memory.

Press any key to continue...
```

Revision history for this message
ali nourzad (anourzad) wrote :
Revision history for this message
Julian Andres Klode (juliank) wrote :

@Craig The limit in Debian is actually much lower than in Ubuntu even, but fixing it there is even harder as that misses a lot more patches.

Revision history for this message
Adrien Nader (adrien-n) wrote :

I put together some notes and work-arounds in order to provide a simpler reference for people hitting this issue. I didn't test everything below but nothing should be risky.

# Summary

Grub attempts to read the initrd into a memory location that is too small.

This issue is caused by a combination of several factors:
- Grub not setting aside enough space for the initrd in memory and not at the right location,
- Initrds having grown due to holding more modules and more firmware. This is especially the case with the nvidia proprietary driver.
- Typical screen resolutions and their associated buffers in memory having increased.

A related issue is not having enough space in /boot to hold enough initrds.

# Solutions

There are four ways to tackle these issues:
- Patch grub,
- Have fewer modules,
- Compress initrds more,
- Lower screen resolution at boot.

## Patch grub
This is being worked on. It’s the trickier option so I’m not going to provide details but you can find that in the discussion above.

## Have fewer modules
The default configuration creates initrds that are meant to be universal in order to accommodate as many hardware variants as possible. Unsurprisingly this makes initrds much bigger.

It is possible to change the value of MODULES to ‘dep’ in /etc/initramfs-tools/initramfs.conf . This will drastically reduce the initrd size in virtually every configuration at the cost of not supporting during boot hardware that is not plugged when the initrd generation takes place.

## Compress initrds more
Change the value of COMPRESS in /etc/initramfs-tools/initramfs.conf .
In terms of compression level, you will have something along the lines of the following:
    lz4: 74066711
    zstd: 54260379
    gzip: 53829310
    xz: 29665544

NB: these values are only valid on one specific machine and configuration; they are only meant to give an idea of compression ratio that can be obtained but the initrd uses MODULES=dep as outlined above.

With a small hack, it is also possible to make xz compression much faster at a small cost in compression by adding the following at the end of initramfs.conf:
    export XZ_OPT='--lzma2=preset=0,dict=8M'

In the future, it will be possible to directly set compression levels for every compression method.

## Lower screen resolution at boot
You can use a lower resolution screen when booting.

You can also edit /etc/default/grub and use the following:
    GRUB_GFXMODE=640x480
Remember to run update-grub afterwards.
If you need to do it at boot-time, you will use GFXMODE instead (no ‘GRUB_’ prefix).

Revision history for this message
jeremyszu (os369510) wrote :

Is it possible if we have an overwrite in debian/rule by using something like DEB_BUILD_ARCH to specify higher compression rate for amd64?

Revision history for this message
Craig Hesling (hesling-craig) wrote :

Ah, didn't realize Debian had a different limit. Thanks @juliank.

@anourzad, I'm not sure I would want to start adding custom steps to the kernel update path. I'm much more inclined to add an /etc option for a supported automatic mechanism, like compression or module selection.

@adrien-n, great notes in #125, I'm sure this will help others.

description: updated
Revision history for this message
Adrien Nader (adrien-n) wrote :

Jeremy, there are duplicate firmware files. Replacing duplicates with symlinks is probably the easiest and most efficient way to improve the situation.

I get the following:

> % jdupes -mrS /lib/firmware
> Scanning: 2830 files, 286 items (in 1 specified)
> 405 duplicate files (in 212 sets), occupying 37 MB

With "jdupes -rl", "tar c /lib/firmware | zstd -1" drops from 428MB to 411MB. The effect on initrds is maybe slightly lower iirc but it's still noticeable and it should be very easy and safe for us.

I can't tell exactly how many more people will be able to boot with that change but I think the cost is small enough that it's worth trying. Worst case, every kernel update is faster.

tags: added: originate-from-1998320 somerville
Revision history for this message
Julian Andres Klode (juliank) wrote :

I have now picked all the rhboot patches in 2.06-2ubuntu16~ppa1 in https://launchpad.net/~ubuntu-uefi-team/+archive/ubuntu/ppa/+packages and will boot that before uploading to the archive.

This *should* allow initrds over 4GB but obviously bugs could be there. The important bit is that we are now close to the rhboot branch again in that file such that the whole thing is maintainable to some extent at least.

Once this is uploaded to lunar people can test this easily on secure boot systems without having to install PPA signing keys in MOK.

I'm gonna see if I can run some tests myself and reproduce this.

Also it turns out it fails to build the i386-efi target due to pointer size mismatches.

Revision history for this message
Nintendo Maniac 64 (nm64) wrote (last edit ):

I'm insanely late on this, but I couldn't help but notice that nobody mentioned Ventoy (at least ctrl+f doesn't return any results in the full activity log for "ventoy")

Since installing via Rufus set to ISO mode has been a suggested work-around, wouldn't this by definition mean that using Ventoy would "just work"?

Normally I would test this myself and report back, but I'm not actually running into this issue myself, so...

Link for those that aren't aware of what Ventoy is:
https://github.com/ventoy/Ventoy

Changed in grub2-unsigned (Ubuntu):
status: Triaged → Fix Committed
Changed in grub2-signed (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
ybdjkfd (ybdjkfd) wrote (last edit ):

Introduction: The original workaround fixes I posted in #41 and #45 (i.e., the workarounds I found to get things to boot), after upgrading a kernon on 22.10, I now have an updated issue.

Issue now: I still have the workarounds active (after the upgrade) so that the compression and module dependency setups are fixed to make a smaller initrd. After an upgrade to the newest kernel 5.19.0-26-generic, I get on grub:

error: out of memory
Press any key to boot . . .

Pressing a key will normally boot forward this time thankfully. My /boot/initrd.img-5.19.0-26-generic size is 131 Mb. This happens only with this new kernel. The previous 5.19.0-23 one did not have this similar error show and is 49 Mb per the workaround setups above.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Hi ybdjkfd I don't really understand what you are doing and how it's relevant. This issue should be fixed in lunar-proposed 2.06-2ubuntu16, please only add comments if you have meaningful insights regarding the verification of that update.

Revision history for this message
ybdjkfd (ybdjkfd) wrote (last edit ):

Hi juliank, I understand completely you want me to have insights related to verification of the update in lunar-proposed and that I am confusing you with my #131 post.

I have edited it for clarity. I hope it helps people who need the workarounds until lunar or backported fixes are released

Revision history for this message
Julian Andres Klode (juliank) wrote :

The term you're looking for is workaround, not triage or fixes. Triage is the process of understanding the bug and marking it the correct things.

Not using the same language as everyone else and writing very long comments makes it hard to understand.

First I thought you reported an out of memory issue on Pis. Then I thought you were saying that the fix doesn't help. It took me a long while to realize it was completely irrelevant for the process of resolving that bug.

If the initrd grows significantly please open a separate bug against linux so that is tracked - maybe something went wrong - and leave a comment here.

Revision history for this message
ybdjkfd (ybdjkfd) wrote (last edit ):

juliank, edited and removed more notes as you advised. I hope my edits per your advice help clarify things and help anyone to see if a new bug is necessary to report. Thanks for your input.

jeremyszu (os369510)
description: updated
description: updated
Revision history for this message
Chaim Eliyah (chaimeliyah) wrote (last edit ):

Can confirm on 22.10, Lenovo Yoga 720 (4k display). Edit: looks like #125 has a comprehensive list of workarounds. Edit2: Holy moly,

-rw-r--r-- 1 root root 115M Dec 18 03:33 initrd.img-5.19.0-26-generic
-rw-r--r-- 1 root root 22M Dec 18 05:08 initrd.img-5.19.0-26-generic.new

Edit3: This definitely got me to (initramfs), however the root volume wasn't available. (crypt+LVM) To boot, I had to:

exit

(because it never shows you the error until you do this)

cryptsetup open /dev/nvme0n1p3 root
lvm lvscan

and voila. I don't know if this is related to "MODULES=dep" (would this keep it from opening the crypt?)... but it's been a day, will test and report back later. :greenbeer:

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package grub2-unsigned - 2.06-2ubuntu16

---------------
grub2-unsigned (2.06-2ubuntu16) lunar; urgency=medium

  * Cherry-pick all memory patches from rhboot
    - Allocate initrd > 4 GB (LP: #1842320)
    - Allocate kernels as code, not data (needed for newer firmware)
  * ubuntu: Fix casts on i386-efi target
  * Cherry-pick all the 2.12 memory management changes (LP: #1842320)
  * Allocate executables as CODE, not DATA in chainloader and arm64
  * Source package generated from src:grub2 using make -f ./debian/rules
    generate-grub2-unsigned

 -- Julian Andres Klode <email address hidden> Fri, 09 Dec 2022 17:11:44 +0100

Changed in grub2-unsigned (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Rex Tsai (chihchun) wrote :

Hi, There are a few OEM platforms that also requires these memory patches in grub2 on jammy, could we also port 2.06-2ubuntu16 back to jammy?

tags: added: originate-from-1998995
Cyrus Lien (cyruslien)
tags: added: originate-from-2000298
Revision history for this message
Aaron Honeycutt (aaronhoneycutt) wrote :

I second backporting this to jammy at least as it is an LTS.

Revision history for this message
iMac (imac-netstatz) wrote :

Backporting to LTS is a good idea IMHO. A few additional use cases to consider

- Users on the threshold who experience slight initrd image growth due to any module update

- Users who thought they might be able to continue just using older images may fail to recognize grub will eventually remove those older images during regular updates.

- Users that take their laptops on the road, but then plug them into their dock with larger displays to get the real work done only to have a boot failure. And yet when they take it into Best Buy it just works somehow.

Revision history for this message
Julian Andres Klode (juliank) wrote :

The entire upload will be uploaded to all older releases but we do have a security upload already in the unapproved queues that should go out first, but that needs approving those first, them passing the SRU verification, a resigning against the new signing key the new shim needs, and finally someone to release them. So that the security update can be released to security pocket.

This won't be ready before Monday the week after next week at the earliest (7 day testing period in proposed).

People that can approve are still out and only return next week, so I don't think that's entirely realistic we get the existing uploads into proposed on Monday.

If we could get them in by Thursday they could go out Monday the 23rd and we may be able to then upload this fix and release it on like 31st or so.

We're strictly limited by the point release - it needs to be ready for that.

Changed in grub2-signed (Ubuntu):
milestone: none → ubuntu-22.04.2
Changed in grub2-unsigned (Ubuntu):
milestone: none → ubuntu-22.04.2
Revision history for this message
Madpower (madzohan) wrote (last edit ):

Hi guys, I want to share my success story:
It happened after I've updated to CUDA repo's nvidia drivers :) my `initrd.img-5.15.0-58-generic` (`ls -lah /boot`) was about 192M and using tuning option of the `/etc/initramfs-tools/initramfs.conf` file I've decreased it (`sudo update-initramfs -c -k 5.15.0-58-generic`) to 116M and now it works =)

my `/etc/initramfs-tools/initramfs.conf`

```
#
# initramfs.conf
# Configuration file for mkinitramfs(8). See initramfs.conf(5).
#
# Note that configuration options from this file can be overridden
# by config files in the /etc/initramfs-tools/conf.d directory.

#
# MODULES: [ most | netboot | dep | list ]
#
# most - Add most filesystem and all harddrive drivers.
#
# dep - Try and guess which modules to load.
#
# netboot - Add the base modules, network modules, but skip block devices.
#
# list - Only include modules from the 'additional modules' list
#

# original
#MODULES=most

# patch
MODULES=dep

#
# BUSYBOX: [ y | n | auto ]
#
# Use busybox shell and utilities. If set to n, klibc utilities will be used.
# If set to auto (or unset), busybox will be used if installed and klibc will
# be used otherwise.
#

BUSYBOX=auto

#
# COMPRESS: [ gzip | bzip2 | lz4 | lzma | lzop | xz | zstd ]
#

# original
# COMPRESS=zstd

COMPRESS=xz

#
# DEVICE: ...
#
# Specify a specific network interface, like eth0
# Overridden by optional ip= or BOOTIF= bootarg
#

DEVICE=

#
# NFSROOT: [ auto | HOST:MOUNT ]
#

NFSROOT=auto

#
# RUNSIZE: ...
#
# The size of the /run tmpfs mount point, like 256M or 10%
# Overridden by optional initramfs.runsize= bootarg
#

RUNSIZE=10%

#
# FSTYPE: ...
#
# The filesystem type(s) to support, or "auto" to use the current root
# filesystem type
#

FSTYPE=auto

# patch
export XZ_OPT='--lzma2=preset=0,dict=8M'
```

Hope it will help you (and don't forget about `sudo update-initramfs -c -k YOUR_VERSION-generic`). Keep yourself safe (from kernel panics too) and Glory to Ukraine!

Revision history for this message
ybdjkfd (ybdjkfd) wrote (last edit ):

@madzohan Congrats to you. It is nice to get things working for now.
 From my comment around #41 above and related workarounds from there downward, it seems you tried a different compression scheme and found a smaller size in this case from zstd to xz.

I looked over your settings and options for xz compression, but also reading the man pages for XZ it shows that it is already at a -6 setting by default that makes the dictionary size 8M and uses lzma2 compression. XZ settings are already at default in /etc/sbin/mkinitramfs .

With your options it was a few MB larger than from xz default in my setup. I tried both xz settings. Finally, I switched back to zstd and my compression changes I made in /etc/sbin/mkinitramfs as my workaround. I got a good compression size, but people note online that zstd has great read speeds over xz no matter the compression level. So I decided to stay at zstd with only a 7 MB difference: from 125 MB on default xz to 131 MB on zstd with -q 22 compression in /etc/sbin/mkinitramfs .

Revision history for this message
Adrien Nader (adrien-n) wrote :

The terrible thing with compression is how we know of no universal rule. I'm sure you can even find non-pathological cases where lz4 compresses better than zpaq (and does so 1000000 times faster). And that's without taking I/O into account (or filters).

An important thing to keep in mind here is that zstd currently uses its -1 preset while xz uses its default of -6. That means that these two tools which are often in the same ballpark are configured at vastly different compression/speed setting. What's important is that you get something that works for your setup and as I said above: we don't have a rule for optimal settings (and it's probably impossible to find one).

One last note: xz decompression speed depends on the compressed size, not uncompressed size. This means that a more-compressed file decompresses faster. This might be the same with zstd since they're related compressors but I'm not 100% sure.

Revision history for this message
Julian Andres Klode (juliank) wrote :

This fix will be included in grub2-unsigned/2.06-2ubuntu14.1 for kinetic, jammy, focal, bionic. Binaries have been built in https://launchpad.net/~ubuntu-uefi-team/+archive/ubuntu/ppa/+packages for kinetic.

Signing request: https://answers.launchpad.net/canonical-signing-jobs/+question/704589

no longer affects: initramfs-tools (Ubuntu Kinetic)
no longer affects: initramfs-tools (Ubuntu Jammy)
no longer affects: initramfs-tools (Ubuntu Focal)
no longer affects: initramfs-tools (Ubuntu Bionic)
no longer affects: linux (Ubuntu)
no longer affects: initramfs-tools (Ubuntu)
no longer affects: linux (Ubuntu Bionic)
no longer affects: linux (Ubuntu Focal)
no longer affects: linux (Ubuntu Jammy)
no longer affects: linux (Ubuntu Kinetic)
description: updated
Revision history for this message
Julian Andres Klode (juliank) wrote :

All grubs uploaded to proposed unapproved queues.

Changed in grub2-signed (Ubuntu Bionic):
status: New → In Progress
Changed in grub2-signed (Ubuntu Focal):
status: New → In Progress
Changed in grub2-signed (Ubuntu Jammy):
status: New → In Progress
Changed in grub2-signed (Ubuntu Kinetic):
status: New → In Progress
Changed in grub2-unsigned (Ubuntu Bionic):
status: New → In Progress
Changed in grub2-unsigned (Ubuntu Kinetic):
status: New → In Progress
Changed in grub2-unsigned (Ubuntu Jammy):
status: New → In Progress
Changed in grub2-unsigned (Ubuntu Focal):
status: New → In Progress
Displaying first 40 and last 40 comments. View all 146 comments or add a comment.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.