| 2019-09-02 12:55:55 |
Gordon Mckeown |
bug |
|
|
added bug |
| 2019-09-02 12:55:55 |
Gordon Mckeown |
attachment added |
|
IMG_20190902_131055.jpg https://bugs.launchpad.net/bugs/1842320/+attachment/5286212/+files/IMG_20190902_131055.jpg |
|
| 2019-09-02 13:00:15 |
Gordon Mckeown |
description |
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
ii linux-generic 5.2.0.15.16 amd64 Complete Generic Linux kernel and headers
ii linux-headers-5.2.0-15 5.2.0-15.16 all Header files related to Linux kernel version 5.2.0
ii linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64 Linux kernel headers for version 5.2.0 on 64 bit x86 SMP
ii linux-headers-generic 5.2.0.15.16 amd64 Generic Linux kernel headers
ii linux-image-5.0.0-25-generic 5.0.0-25.26 amd64 Signed kernel image generic
ii linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64 Signed kernel image generic
ii linux-image-generic 5.2.0.15.16 amd64 Generic Linux kernel image
ii linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64 Linux kernel extra modules for version 5.0.0 on 64 bit x86 SMP
ii linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64 Linux kernel extra modules for version 5.2.0 on 64 bit x86 SMP
ii linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64 Linux kernel extra modules for version 5.0.0 on 64 bit x86 SMP
ii linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64 Linux kernel extra modules for version 5.2.0 on 64 bit x86 SMP
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4. |
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4. |
|
| 2019-09-02 13:15:07 |
Paul White |
affects |
ubuntu |
linux (Ubuntu) |
|
| 2019-09-02 13:30:07 |
Ubuntu Kernel Bot |
linux (Ubuntu): status |
New |
Incomplete |
|
| 2019-09-02 13:30:09 |
Ubuntu Kernel Bot |
tags |
|
disco |
|
| 2019-09-02 14:38:48 |
Gordon Mckeown |
tags |
disco |
apport-collected disco eoan |
|
| 2019-09-02 14:38:49 |
Gordon Mckeown |
description |
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4. |
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2019-09-02 14:38:49 |
Gordon Mckeown |
attachment added |
|
AlsaInfo.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286215/+files/AlsaInfo.txt |
|
| 2019-09-02 14:38:51 |
Gordon Mckeown |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286216/+files/CRDA.txt |
|
| 2019-09-02 14:38:52 |
Gordon Mckeown |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286217/+files/CurrentDmesg.txt |
|
| 2019-09-02 14:38:53 |
Gordon Mckeown |
attachment added |
|
IwConfig.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286218/+files/IwConfig.txt |
|
| 2019-09-02 14:38:55 |
Gordon Mckeown |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286219/+files/Lspci.txt |
|
| 2019-09-02 14:38:57 |
Gordon Mckeown |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286220/+files/ProcCpuinfo.txt |
|
| 2019-09-02 14:38:58 |
Gordon Mckeown |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286221/+files/ProcCpuinfoMinimal.txt |
|
| 2019-09-02 14:39:01 |
Gordon Mckeown |
attachment added |
|
ProcEnviron.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286222/+files/ProcEnviron.txt |
|
| 2019-09-02 14:39:02 |
Gordon Mckeown |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286223/+files/ProcInterrupts.txt |
|
| 2019-09-02 14:39:03 |
Gordon Mckeown |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286224/+files/ProcModules.txt |
|
| 2019-09-02 14:39:05 |
Gordon Mckeown |
attachment added |
|
PulseList.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286225/+files/PulseList.txt |
|
| 2019-09-02 14:39:06 |
Gordon Mckeown |
attachment added |
|
RfKill.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286226/+files/RfKill.txt |
|
| 2019-09-02 14:39:07 |
Gordon Mckeown |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286227/+files/UdevDb.txt |
|
| 2019-09-02 14:39:10 |
Gordon Mckeown |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5286228/+files/WifiSyslog.txt |
|
| 2019-09-02 14:41:26 |
Gordon Mckeown |
linux (Ubuntu): status |
Incomplete |
Confirmed |
|
| 2019-09-27 15:30:22 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto Murata |
| 2019-10-01 08:03:21 |
Nobuto Murata |
removed subscriber Nobuto Murata |
|
|
|
| 2019-10-19 09:56:19 |
Steven Mackenzie |
attachment added |
|
Kernel panic screenshot https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5298357/+files/IMG_20191019_174025.jpg |
|
| 2019-12-08 10:46:10 |
Joost Cassee |
bug |
|
|
added subscriber Joost Cassee |
| 2019-12-29 00:05:48 |
Terry Dawson |
bug |
|
|
added subscriber Terry Dawson |
| 2019-12-29 00:06:56 |
Terry Dawson |
attachment added |
|
AlsaInfo.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316142/+files/AlsaInfo.txt |
|
| 2019-12-29 00:06:58 |
Terry Dawson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316143/+files/CurrentDmesg.txt |
|
| 2019-12-29 00:07:00 |
Terry Dawson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316144/+files/Lspci.txt |
|
| 2019-12-29 00:07:02 |
Terry Dawson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316145/+files/Lsusb.txt |
|
| 2019-12-29 00:07:04 |
Terry Dawson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316146/+files/ProcCpuinfo.txt |
|
| 2019-12-29 00:07:06 |
Terry Dawson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316147/+files/ProcCpuinfoMinimal.txt |
|
| 2019-12-29 00:07:08 |
Terry Dawson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316148/+files/ProcInterrupts.txt |
|
| 2019-12-29 00:07:10 |
Terry Dawson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316149/+files/ProcModules.txt |
|
| 2019-12-29 00:07:12 |
Terry Dawson |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316150/+files/UdevDb.txt |
|
| 2019-12-29 00:07:15 |
Terry Dawson |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1842320/+attachment/5316151/+files/WifiSyslog.txt |
|
| 2021-01-21 21:10:13 |
Ben Hoyt |
bug |
|
|
added subscriber Ben Hoyt |
| 2021-01-21 21:39:10 |
Matthew Ruffell |
bug |
|
|
added subscriber Matthew Ruffell |
| 2022-04-29 08:34:16 |
Joost Cassee |
removed subscriber Joost Cassee |
|
|
|
| 2022-04-29 10:31:09 |
Chris Guiver |
tags |
apport-collected disco eoan |
apport-collected jammy |
|
| 2022-05-22 13:26:16 |
sudodus |
bug |
|
|
added subscriber sudodus |
| 2022-05-31 04:33:01 |
jeremyszu |
bug |
|
|
added subscriber OEM Solutions Group: Engineers |
| 2022-05-31 04:33:05 |
jeremyszu |
tags |
apport-collected jammy |
apport-collected jammy jiayi oem-priority originate-from-1972964 |
|
| 2022-05-31 04:52:13 |
jeremyszu |
oem-priority: assignee |
|
jeremyszu (os369510) |
|
| 2022-05-31 04:52:18 |
jeremyszu |
oem-priority: importance |
Undecided |
Critical |
|
| 2022-05-31 04:52:20 |
jeremyszu |
oem-priority: status |
New |
In Progress |
|
| 2022-05-31 05:07:01 |
jeremyszu |
bug task added |
|
initramfs-tools (Ubuntu) |
|
| 2022-05-31 05:09:15 |
jeremyszu |
bug task added |
|
grub2-signed (Ubuntu) |
|
| 2022-06-01 07:01:17 |
jeremyszu |
bug watch added |
|
http://savannah.gnu.org/bugs/?61058 |
|
| 2022-06-01 07:01:17 |
jeremyszu |
bug task added |
|
grub |
|
| 2022-06-09 07:45:34 |
jeremyszu |
tags |
apport-collected jammy jiayi oem-priority originate-from-1972964 |
apport-collected jammy jellyfish-edge-staging jiayi oem-priority originate-from-1972964 |
|
| 2022-06-25 04:22:05 |
jeremyszu |
attachment added |
|
Screenshot from 2022-06-25 12-15-27.png https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5599659/+files/Screenshot%20from%202022-06-25%2012-15-27.png |
|
| 2022-06-26 19:46:32 |
Launchpad Janitor |
grub2-signed (Ubuntu): status |
New |
Confirmed |
|
| 2022-06-26 19:46:32 |
Launchpad Janitor |
initramfs-tools (Ubuntu): status |
New |
Confirmed |
|
| 2022-06-28 14:08:12 |
jeremyszu |
attachment added |
|
0129-Replace-the-memory-allocation-by-using-the-macro.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5600192/+files/0129-Replace-the-memory-allocation-by-using-the-macro.patch |
|
| 2022-06-28 14:08:35 |
jeremyszu |
attachment added |
|
0130-x86-efi-Make-our-own-allocator-for-kernel-stuff.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5600193/+files/0130-x86-efi-Make-our-own-allocator-for-kernel-stuff.patch |
|
| 2022-06-28 14:09:00 |
jeremyszu |
attachment added |
|
0131-x86-efi-Allow-initrd-params-cmdline-allocations-abov.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5600194/+files/0131-x86-efi-Allow-initrd-params-cmdline-allocations-abov.patch |
|
| 2022-06-28 14:09:27 |
jeremyszu |
attachment added |
|
0132-x86-efi-Use-bounce-buffers-for-reading-to-addresses-.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5600195/+files/0132-x86-efi-Use-bounce-buffers-for-reading-to-addresses-.patch |
|
| 2022-06-28 14:09:52 |
jeremyszu |
attachment added |
|
0133-Fix-wrong-print-of-types.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5600196/+files/0133-Fix-wrong-print-of-types.patch |
|
| 2022-06-28 14:59:13 |
jeremyszu |
description |
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects, in many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
…
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages()
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
Any method to grow the initramfs, such as install nvidia-driver.
If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in
X86_64 qemu with
60M initramfs + 5.15.0-37-generic kernel
565M initramfs + 5.17.0-1011-oem kernel
Amd64 HP mobile workstation with
65M initramfs + 5.15.0-39-generic kernel
771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system. The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”. Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev . After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2022-06-28 15:02:50 |
jeremyszu |
description |
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects, in many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
…
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages()
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
Any method to grow the initramfs, such as install nvidia-driver.
If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in
X86_64 qemu with
60M initramfs + 5.15.0-37-generic kernel
565M initramfs + 5.17.0-1011-oem kernel
Amd64 HP mobile workstation with
65M initramfs + 5.15.0-39-generic kernel
771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system. The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”. Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev . After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev .
After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2022-06-28 15:03:09 |
jeremyszu |
oem-priority: status |
In Progress |
Triaged |
|
| 2022-06-28 16:32:34 |
Ubuntu Foundations Team Bug Bot |
tags |
apport-collected jammy jellyfish-edge-staging jiayi oem-priority originate-from-1972964 |
apport-collected jammy jellyfish-edge-staging jiayi oem-priority originate-from-1972964 patch |
|
| 2022-06-28 16:32:37 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Terry Rudd |
| 2022-06-28 16:33:05 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Review Team |
| 2022-06-28 22:47:58 |
jeremyszu |
oem-priority: status |
Triaged |
In Progress |
|
| 2022-06-29 01:58:42 |
jeremyszu |
oem-priority: status |
In Progress |
Triaged |
|
| 2022-07-19 06:10:03 |
Daniel van Vugt |
summary |
Out of Memory on boot with 5.2.0 kernel |
Out of Memory on boot |
|
| 2022-07-19 06:10:06 |
Daniel van Vugt |
bug |
|
|
added subscriber Daniel van Vugt |
| 2022-07-19 06:10:17 |
Daniel van Vugt |
tags |
apport-collected jammy jellyfish-edge-staging jiayi oem-priority originate-from-1972964 patch |
apport-collected jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 patch |
|
| 2022-07-19 06:44:41 |
Daniel van Vugt |
grub2-signed (Ubuntu): importance |
Undecided |
Critical |
|
| 2022-07-19 06:44:44 |
Daniel van Vugt |
initramfs-tools (Ubuntu): importance |
Undecided |
Critical |
|
| 2022-07-19 06:44:47 |
Daniel van Vugt |
linux (Ubuntu): importance |
Undecided |
Critical |
|
| 2022-07-19 06:45:48 |
Daniel van Vugt |
summary |
Out of Memory on boot |
Can't boot: "error: out of memory." immediately after the grub menu |
|
| 2022-08-03 08:44:19 |
Steven Mackenzie |
removed subscriber Steven Mackenzie |
|
|
|
| 2022-08-04 06:50:36 |
Peter-and-Crazybear |
bug |
|
|
added subscriber Peter-and-Crazybear |
| 2022-08-08 21:28:11 |
Rob Cork |
bug |
|
|
added subscriber Rob Cork |
| 2022-08-09 02:51:54 |
jeremyszu |
attachment added |
|
0129-Try-to-pick-better-locations-for-kernel-and-initrd.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607429/+files/0129-Try-to-pick-better-locations-for-kernel-and-initrd.patch |
|
| 2022-08-09 02:52:22 |
jeremyszu |
attachment added |
|
0130-x86-efi-Use-bounce-buffers-for-reading-to-addresses-.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607431/+files/0130-x86-efi-Use-bounce-buffers-for-reading-to-addresses-.patch |
|
| 2022-08-09 02:52:49 |
jeremyszu |
attachment added |
|
0131-x86-efi-Re-arrange-grub_cmd_linux-a-little-bit.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607432/+files/0131-x86-efi-Re-arrange-grub_cmd_linux-a-little-bit.patch |
|
| 2022-08-09 02:53:21 |
jeremyszu |
attachment added |
|
0132-x86-efi-Make-our-own-allocator-for-kernel-stuff.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607433/+files/0132-x86-efi-Make-our-own-allocator-for-kernel-stuff.patch |
|
| 2022-08-09 02:54:05 |
jeremyszu |
attachment added |
|
0133-x86-efi-Allow-initrd-params-cmdline-allocations-abov.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607434/+files/0133-x86-efi-Allow-initrd-params-cmdline-allocations-abov.patch |
|
| 2022-08-09 02:54:32 |
jeremyszu |
attachment added |
|
0134-linuxefi-fail-kernel-validation-without-shim-protoco.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607435/+files/0134-linuxefi-fail-kernel-validation-without-shim-protoco.patch |
|
| 2022-08-09 02:59:18 |
jeremyszu |
attachment added |
|
0135-Fix-4GB-memory-be-filtered-out-by-filter_memory_map.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/+attachment/5607436/+files/0135-Fix-4GB-memory-be-filtered-out-by-filter_memory_map.patch |
|
| 2022-08-16 10:36:28 |
Steven Clarkson |
bug |
|
|
added subscriber Steven Clarkson |
| 2022-08-17 20:11:48 |
John Wiggins |
bug |
|
|
added subscriber John Wiggins |
| 2022-08-18 10:03:09 |
robinkooli |
bug |
|
|
added subscriber robinkooli |
| 2022-08-22 14:45:39 |
Mauricio Faria de Oliveira |
bug |
|
|
added subscriber Mauricio Faria de Oliveira |
| 2022-09-01 09:59:00 |
ali nourzad |
bug |
|
|
added subscriber ali nourzad |
| 2022-09-02 11:23:04 |
jeremyszu |
bug |
|
|
added subscriber Heinrich Schuchardt |
| 2022-09-15 17:58:58 |
Brian Murray |
tags |
apport-collected jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 patch |
apport-collected foundations-todo jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 patch |
|
| 2022-09-20 12:36:37 |
Fabio Augusto Miranda Martins |
bug |
|
|
added subscriber Fabio Augusto Miranda Martins |
| 2022-09-21 19:50:37 |
Mar Kus |
bug |
|
|
added subscriber Mar Kus |
| 2022-09-22 01:26:03 |
Jay Vosburgh |
bug |
|
|
added subscriber Jay Vosburgh |
| 2022-10-04 00:00:09 |
Steve Langasek |
grub2-signed (Ubuntu): status |
Confirmed |
Triaged |
|
| 2022-10-04 00:01:45 |
Steve Langasek |
initramfs-tools (Ubuntu): status |
Confirmed |
Won't Fix |
|
| 2022-10-05 16:19:39 |
Dan Bungert |
bug |
|
|
added subscriber Ubuntu Foundations Bugs |
| 2022-10-20 12:15:04 |
Julian Andres Klode |
bug task added |
|
grub2-unsigned (Ubuntu) |
|
| 2022-10-20 12:15:34 |
Julian Andres Klode |
grub2-unsigned (Ubuntu): status |
New |
Triaged |
|
| 2022-10-20 12:15:45 |
Julian Andres Klode |
grub2-unsigned (Ubuntu): importance |
Undecided |
Critical |
|
| 2022-10-27 10:16:05 |
Andy Chi |
tags |
apport-collected foundations-todo jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 patch |
apport-collected foundations-todo jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 patch stella |
|
| 2022-11-01 04:12:02 |
iMac |
bug |
|
|
added subscriber iMac |
| 2022-11-03 10:13:31 |
Julian Andres Klode |
tags |
apport-collected foundations-todo jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 patch stella |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 patch stella |
|
| 2022-11-03 11:30:48 |
Yohn Chrichton |
bug |
|
|
added subscriber Yohn Chrichton |
| 2022-11-07 15:11:21 |
Julian Andres Klode |
grub2-unsigned (Ubuntu): assignee |
|
Julian Andres Klode (juliank) |
|
| 2022-11-07 15:11:23 |
Julian Andres Klode |
grub2-signed (Ubuntu): assignee |
|
Julian Andres Klode (juliank) |
|
| 2022-11-29 17:47:17 |
Brian Murray |
description |
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev .
After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev .
After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2022-12-02 02:13:56 |
Kai-Chuan Hsieh |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 patch stella |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 patch somerville stella |
|
| 2022-12-02 17:19:33 |
Joe Barnett |
bug |
|
|
added subscriber Joe Barnett |
| 2022-12-09 16:51:03 |
Julian Andres Klode |
grub2-unsigned (Ubuntu): status |
Triaged |
Fix Committed |
|
| 2022-12-09 16:51:06 |
Julian Andres Klode |
grub2-signed (Ubuntu): status |
Triaged |
Fix Committed |
|
| 2022-12-13 03:36:57 |
m1st0 |
bug |
|
|
added subscriber ybdjkfd |
| 2022-12-14 11:02:09 |
jeremyszu |
description |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffff
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fffffff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fffffff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fffffff
#endif
```
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* Upstream grub2 bug #61058
https://savannah.gnu.org/bugs/index.php?61058
* Test PPA: https://launchpad.net/~os369510/+archive/ubuntu/lp1842320
* Test grubx64.efi: https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320
* Test source code: https://github.com/os369510/grub2/tree/lp1842320
* If you built the package, then test grubx64.efi is under “obj/monolithic/grub-efi-amd64/grubx64.efi”, in my case: `/var/cache/pbuilder/build/276481/build/grub2-2.06/obj/monolithic/grub-efi-amd64/grubx64.efi`
* My build command: `sudo PBSHELL=1 pbuilder build --hookdir ~/hook-dir ubuntu-grub/grub2_2.06-2ubuntu7+jeremydev2.dsc 2>&1 | tee build.log`
* My qemu command: `qemu-system-x86_64 -bios edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -hda Templates/grub.qcow2 -m 6G -vga cirrus -smp 8 -machine type=q35,accel=kvm -cpu host -enable-kvm -boot menu=on` (I built an edk2 binary with debugging log)
* You can use my grubx64.efi with debug symbols from https://people.canonical.com/~jeremysu/lp1842320/grubx64.efi.lp1842320-dev-with-debug-symbols and source code is from https://github.com/os369510/grub2/tree/jeremy-dev .
After built the package from source code, then you can use gdb to attach the qemu session as:
```
ubuntu@ubuntu-HP-ZBook-Fury-16-G9-Mobile-Workstation-PC [ /var/cache/pbuilder/build/35354/tmp/buildd/grub2-2.06/obj/grub-efi-amd64/grub-core ]
$ gdb -x gdb_grub # with “add-symbol-file kernel.img ${address}
```
The address above can read from qemu serial port and found the last
“Loading driver at 0x000xxxxxxxxxx EntryPoint=0x000xxxxxxxabc”
In above case, fill “0x000xxxxxxxabc” to ${address}.
[1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf
[2] https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/4_general_driver_design_guidelines/readme.2/423_use_uefi_memory_allocation_services
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
All working well.
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2022-12-14 11:03:01 |
jeremyszu |
description |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
All working well.
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2022-12-15 08:41:07 |
loukangzhen |
bug |
|
|
added subscriber loukangzhen |
| 2022-12-19 02:08:58 |
Launchpad Janitor |
grub2-unsigned (Ubuntu): status |
Fix Committed |
Fix Released |
|
| 2022-12-20 04:30:19 |
Rex Tsai |
bug |
|
|
added subscriber Rex Tsai |
| 2022-12-20 04:32:50 |
Rex Tsai |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 patch somerville stella |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 patch somerville stella |
|
| 2022-12-23 02:40:01 |
Cyrus Lien |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 patch somerville stella |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella |
|
| 2023-01-06 16:42:22 |
Duarte Rocha |
bug |
|
|
added subscriber Duarte Rocha |
| 2023-01-06 19:25:54 |
Julian Andres Klode |
grub2-signed (Ubuntu): milestone |
|
ubuntu-22.04.2 |
|
| 2023-01-06 19:26:09 |
Julian Andres Klode |
grub2-unsigned (Ubuntu): milestone |
|
ubuntu-22.04.2 |
|
| 2023-01-09 01:13:45 |
Sky |
bug |
|
|
added subscriber Sky |
| 2023-01-10 23:11:42 |
Danky |
bug |
|
|
added subscriber Danky |
| 2023-01-11 05:49:36 |
Rik Mills |
bug |
|
|
added subscriber Rik Mills |
| 2023-01-13 16:07:35 |
Alexandru Cucu |
bug |
|
|
added subscriber Alexandru Cucu |
| 2023-01-30 12:32:38 |
Julian Andres Klode |
nominated for series |
|
Ubuntu Bionic |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
initramfs-tools (Ubuntu Bionic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
linux (Ubuntu Bionic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-signed (Ubuntu Bionic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-unsigned (Ubuntu Bionic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
nominated for series |
|
Ubuntu Jammy |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
initramfs-tools (Ubuntu Jammy) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
linux (Ubuntu Jammy) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-signed (Ubuntu Jammy) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-unsigned (Ubuntu Jammy) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
nominated for series |
|
Ubuntu Focal |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
initramfs-tools (Ubuntu Focal) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
linux (Ubuntu Focal) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-signed (Ubuntu Focal) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-unsigned (Ubuntu Focal) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
nominated for series |
|
Ubuntu Kinetic |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
initramfs-tools (Ubuntu Kinetic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
linux (Ubuntu Kinetic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-signed (Ubuntu Kinetic) |
|
| 2023-01-30 12:32:38 |
Julian Andres Klode |
bug task added |
|
grub2-unsigned (Ubuntu Kinetic) |
|
| 2023-01-30 12:32:52 |
Julian Andres Klode |
bug task deleted |
initramfs-tools (Ubuntu Kinetic) |
|
|
| 2023-01-30 12:32:58 |
Julian Andres Klode |
bug task deleted |
initramfs-tools (Ubuntu Jammy) |
|
|
| 2023-01-30 12:33:06 |
Julian Andres Klode |
bug task deleted |
initramfs-tools (Ubuntu Focal) |
|
|
| 2023-01-30 12:33:14 |
Julian Andres Klode |
bug task deleted |
initramfs-tools (Ubuntu Bionic) |
|
|
| 2023-01-30 12:33:23 |
Julian Andres Klode |
bug task deleted |
linux (Ubuntu) |
|
|
| 2023-01-30 12:33:29 |
Julian Andres Klode |
bug task deleted |
initramfs-tools (Ubuntu) |
|
|
| 2023-01-30 12:33:36 |
Julian Andres Klode |
bug task deleted |
linux (Ubuntu Bionic) |
|
|
| 2023-01-30 12:33:43 |
Julian Andres Klode |
bug task deleted |
linux (Ubuntu Focal) |
|
|
| 2023-01-30 12:33:49 |
Julian Andres Klode |
bug task deleted |
linux (Ubuntu Jammy) |
|
|
| 2023-01-30 12:33:59 |
Julian Andres Klode |
bug task deleted |
linux (Ubuntu Kinetic) |
|
|
| 2023-01-30 12:39:52 |
Julian Andres Klode |
description |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2023-01-30 15:18:00 |
Julian Andres Klode |
grub2-signed (Ubuntu Bionic): status |
New |
In Progress |
|
| 2023-01-30 15:18:03 |
Julian Andres Klode |
grub2-signed (Ubuntu Focal): status |
New |
In Progress |
|
| 2023-01-30 15:18:05 |
Julian Andres Klode |
grub2-signed (Ubuntu Jammy): status |
New |
In Progress |
|
| 2023-01-30 15:18:08 |
Julian Andres Klode |
grub2-signed (Ubuntu Kinetic): status |
New |
In Progress |
|
| 2023-01-30 15:18:21 |
Julian Andres Klode |
grub2-unsigned (Ubuntu Bionic): status |
New |
In Progress |
|
| 2023-01-30 15:18:46 |
Julian Andres Klode |
grub2-unsigned (Ubuntu Kinetic): status |
New |
In Progress |
|
| 2023-01-30 15:18:56 |
Julian Andres Klode |
grub2-unsigned (Ubuntu Jammy): status |
New |
In Progress |
|
| 2023-01-30 15:19:00 |
Julian Andres Klode |
grub2-unsigned (Ubuntu Focal): status |
New |
In Progress |
|
| 2023-01-31 19:21:48 |
Steve Langasek |
grub2-unsigned (Ubuntu Kinetic): status |
In Progress |
Fix Committed |
|
| 2023-01-31 19:21:50 |
Steve Langasek |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2023-01-31 19:21:53 |
Steve Langasek |
bug |
|
|
added subscriber SRU Verification |
| 2023-01-31 19:22:11 |
Steve Langasek |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-kinetic |
|
| 2023-01-31 19:23:42 |
Steve Langasek |
grub2-signed (Ubuntu Kinetic): status |
In Progress |
Fix Committed |
|
| 2023-01-31 19:25:03 |
Steve Langasek |
grub2-unsigned (Ubuntu Jammy): status |
In Progress |
Fix Committed |
|
| 2023-01-31 19:25:22 |
Steve Langasek |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-kinetic |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-jammy verification-needed-kinetic |
|
| 2023-01-31 19:26:18 |
Steve Langasek |
grub2-signed (Ubuntu Jammy): status |
In Progress |
Fix Committed |
|
| 2023-01-31 21:11:21 |
Steve Langasek |
grub2-unsigned (Ubuntu Focal): status |
In Progress |
Fix Committed |
|
| 2023-01-31 21:11:40 |
Steve Langasek |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-jammy verification-needed-kinetic |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-focal verification-needed-jammy verification-needed-kinetic |
|
| 2023-01-31 21:15:12 |
Steve Langasek |
grub2-signed (Ubuntu Focal): status |
In Progress |
Fix Committed |
|
| 2023-01-31 21:23:01 |
Steve Langasek |
grub2-unsigned (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2023-01-31 21:23:20 |
Steve Langasek |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-focal verification-needed-jammy verification-needed-kinetic |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic |
|
| 2023-01-31 21:29:12 |
Steve Langasek |
grub2-signed (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2023-02-01 13:51:42 |
Matthew L. Dailey |
bug |
|
|
added subscriber Matthew L. Dailey |
| 2023-02-01 14:16:57 |
Matthew L. Dailey |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-needed verification-needed-bionic verification-needed-focal verification-needed-kinetic |
|
| 2023-02-01 19:07:42 |
m1st0 |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-needed verification-needed-bionic verification-needed-focal verification-needed-kinetic |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-done-kinetic verification-needed verification-needed-bionic verification-needed-focal |
|
| 2023-02-05 23:40:45 |
Ben Grotenhuis |
removed subscriber Ben Grotenhuis |
|
|
|
| 2023-02-08 11:09:37 |
Julian Andres Klode |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-done-kinetic verification-needed verification-needed-bionic verification-needed-focal |
apport-collected block-proposed-bionic block-proposed-focal foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-done-kinetic verification-needed verification-needed-bionic verification-needed-focal |
|
| 2023-02-13 23:45:05 |
sudodus |
description |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
Today, 2023-02-14, the following link works to find the workarounds:
https://bugs.launchpad.net/oem-priority/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2023-02-15 04:19:25 |
mbott |
bug |
|
|
added subscriber mbott |
| 2023-02-15 11:54:26 |
Julian Andres Klode |
tags |
apport-collected block-proposed-bionic block-proposed-focal foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-jammy verification-done-kinetic verification-needed verification-needed-bionic verification-needed-focal |
apport-collected block-proposed-focal foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-bionic verification-done-jammy verification-done-kinetic verification-needed verification-needed-focal |
|
| 2023-02-15 12:02:37 |
Julian Andres Klode |
tags |
apport-collected block-proposed-focal foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-bionic verification-done-jammy verification-done-kinetic verification-needed verification-needed-focal |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-bionic verification-done-focal verification-done-jammy verification-done-kinetic verification-needed |
|
| 2023-02-16 10:31:19 |
Launchpad Janitor |
grub2-unsigned (Ubuntu Kinetic): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 10:31:23 |
Launchpad Janitor |
grub2-signed (Ubuntu Kinetic): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 10:31:28 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2023-02-16 11:04:20 |
Launchpad Janitor |
grub2-unsigned (Ubuntu Jammy): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 11:04:24 |
Launchpad Janitor |
grub2-signed (Ubuntu Jammy): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 11:10:33 |
Launchpad Janitor |
grub2-unsigned (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 11:10:37 |
Launchpad Janitor |
grub2-signed (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 11:11:48 |
Launchpad Janitor |
grub2-unsigned (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2023-02-16 11:11:52 |
Launchpad Janitor |
grub2-signed (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2023-03-14 00:21:03 |
jeremyszu |
oem-priority: status |
Triaged |
Fix Released |
|
| 2023-03-29 16:14:07 |
Mauricio Faria de Oliveira |
removed subscriber Mauricio Faria de Oliveira |
|
|
|
| 2023-03-29 20:22:13 |
m1st0 |
description |
[Workaround]
Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
Today, 2023-02-14, the following link works to find the workarounds:
https://bugs.launchpad.net/oem-priority/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
[Workaround]
Today, 2023-02-14, the following link works to find the workarounds:
https://bugs.launchpad.net/oem-priority/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 0000000000000000-0000000000086FFF 0000000000000087 000000000000000F
BS_Data 0000000000087000-0000000000087FFF 0000000000000001 000000000000000F
Available 0000000000088000-000000000009EFFF 0000000000000017 000000000000000F
Reserved 000000000009F000-000000000009FFFF 0000000000000001 000000000000000F
Available 0000000000100000-0000000000FFFFFF 0000000000000F00 000000000000000F
LoaderCode 0000000001000000-0000000001021FFF 0000000000000022 000000000000000F
Available 0000000001022000-00000000238A7FFF 0000000000022886 000000000000000F
BS_Data 00000000238A8000-0000000023927FFF 0000000000000080 000000000000000F
Available 0000000023928000-0000000028860FFF 0000000000004F39 000000000000000F
BS_Data 0000000028861000-000000002AB09FFF 00000000000022A9 000000000000000F
LoaderCode 000000002AB0A000-000000002ACF8FFF 00000000000001EF 000000000000000F
BS_Data 000000002ACF9000-000000002B2FAFFF 0000000000000602 000000000000000F
Available 000000002B2FB000-000000002B611FFF 0000000000000317 000000000000000F
BS_Data 000000002B612000-000000002B630FFF 000000000000001F 000000000000000F
Available 000000002B631000-000000002B632FFF 0000000000000002 000000000000000F
BS_Data 000000002B633000-000000002B63CFFF 000000000000000A 000000000000000F
Available 000000002B63D000-000000002B649FFF 000000000000000D 000000000000000F
BS_Data 000000002B64A000-000000002B64EFFF 0000000000000005 000000000000000F
Available 000000002B64F000-000000002B666FFF 0000000000000018 000000000000000F
BS_Data 000000002B667000-000000002D8D5FFF 000000000000226F 000000000000000F
LoaderCode 000000002D8D6000-000000002D8E9FFF 0000000000000014 000000000000000F
BS_Data 000000002D8EA000-000000002D925FFF 000000000000003C 000000000000000F
LoaderCode 000000002D926000-000000002D932FFF 000000000000000D 000000000000000F
BS_Data 000000002D933000-000000002D969FFF 0000000000000037 000000000000000F
BS_Code 000000002D96A000-000000002D973FFF 000000000000000A 000000000000000F
BS_Data 000000002D974000-000000002E377FFF 0000000000000A04 000000000000000F
Available 000000002E378000-000000002E37AFFF 0000000000000003 000000000000000F
...
Reserved 000000003C08B000-000000004192FFFF 00000000000058A5 000000000000000F
ACPI_NVS 0000000041930000-0000000041B2FFFF 0000000000000200 000000000000000F
ACPI_Recl 0000000041B30000-0000000041BFEFFF 00000000000000CF 000000000000000F
BS_Data 0000000041BFF000-0000000041BFFFFF 0000000000000001 000000000000000F
Available 0000000100000000-00000002AB7FFFFF 00000000001AB800 000000000000000F
Reserved 00000000000A0000-00000000000FFFFF 0000000000000060 0000000000000000
Reserved 0000000041C00000-0000000043FFFFFF 0000000000002400 0000000000000000
Reserved 0000000044000000-0000000047FFFFFF 0000000000004000 000000000000000F
Reserved 0000000049400000-00000000495FFFFF 0000000000000200 000000000000000F
Reserved 000000004C000000-000000004FFFFFFF 0000000000004000 0000000000000009
Reserved 0000000050000000-00000000547FFFFF 0000000000004800 0000000000000000
MMIO 00000000C0000000-00000000CFFFFFFF 0000000000010000 8000000000000001
Reserved 00000000FED20000-00000000FED7FFFF 0000000000000060 0000000000000000
MMIO 00000000FF800000-00000000FFFFFFFF 0000000000000800 8000000000001000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory.
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.img-5.17.0-1021-oem # pass
453M /boot/initrd.img-5.17.0-1024-oem # pass
471M /boot/initrd.img-5.17.0-1024-oem # fail
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but it use the original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There are some “#if defined(__x86_64__)” which intent to limit the > 32bits code in i386 system.
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-5.2.0-15 5.2.0-15.16 all
linux-headers-5.2.0-15-generic 5.2.0-15.16 amd64
linux-headers-generic 5.2.0.15.16 amd64
linux-image-5.0.0-25-generic 5.0.0-25.26 amd64
linux-image-5.2.0-15-generic 5.2.0-15.16+signed1 amd64
linux-image-generic 5.2.0.15.16 amd64
linux-modules-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-5.2.0-15-generic 5.2.0-15.16 amd64
linux-modules-extra-5.0.0-25-generic 5.0.0-25.26 amd64
linux-modules-extra-5.2.0-15-generic 5.2.0-15.16 amd64
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/nvme0n1p3_crypt ubuntu-vg lvm2 a-- <475.71g 0
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: gmckeown 1647 F.... pulseaudio
CurrentDesktop: ubuntu:GNOMEDistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.0.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 5.0.0-25.26-generic 5.0.18
RelatedPackageVersions:
linux-restricted-modules-5.0.0-25-generic N/A
linux-backports-modules-5.0.0-25-generic N/A
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAMI:bvrF.25:bd05/17/2019:svnHP:pnHPSpectrex360Convertible13-ae0xx:pvr:rvnHP:rn83B9:rvr56.43:cvnHP:ct31:cvrChassisVersion:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP |
|
| 2023-03-29 21:07:26 |
Marietto |
summary |
Can't boot: "error: out of memory." immediately after the grub menu |
Can't boot: "error: out of memory." immediately after having selected ubuntu on the grub menu if it is populated with a lof of entries. |
|
| 2023-03-29 21:07:41 |
Marietto |
summary |
Can't boot: "error: out of memory." immediately after having selected ubuntu on the grub menu if it is populated with a lof of entries. |
Can't boot: "error: out of memory." immediately after having selected ubuntu on the grub menu if it is populated with a lot of entries. |
|
| 2023-03-30 06:11:19 |
Julian Andres Klode |
summary |
Can't boot: "error: out of memory." immediately after having selected ubuntu on the grub menu if it is populated with a lot of entries. |
Can't boot: "error: out of memory." immediately after the grub menu |
|
| 2023-03-30 14:11:16 |
iMac |
removed subscriber iMac |
|
|
|
| 2023-06-01 13:23:16 |
Julian Andres Klode |
grub2-signed (Ubuntu): status |
Fix Committed |
Fix Released |
|
| 2023-06-02 15:21:44 |
Benjamin Drung |
tags |
apport-collected foundations-todo fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-bionic verification-done-focal verification-done-jammy verification-done-kinetic verification-needed |
apport-collected fr-2934 jammy jellyfish-edge-staging jiayi kinetic oem-priority originate-from-1972964 originate-from-1994098 originate-from-1998320 originate-from-1998995 originate-from-2000298 patch somerville stella verification-done-bionic verification-done-focal verification-done-jammy verification-done-kinetic verification-needed |
|
| 2023-06-02 16:51:14 |
Rob Cork |
removed subscriber Rob Cork |
|
|
|
| 2024-01-02 10:32:35 |
Benjamin Drung |
removed subscriber Ubuntu Foundations Bugs |
|
|
|
