armhf VMs broken with more than 3GB of RAM

Bug #2043471 reported by Skia
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
edk2 (Ubuntu)
Confirmed
Low
Mate Kukri
Mantic
Confirmed
Low
Mate Kukri
Noble
Confirmed
Low
Mate Kukri
qemu (Ubuntu)
Incomplete
Undecided
Unassigned

Bug Description

Hi,

During some autopkgtest debugging, we found that we were unable to boot an armhf virtual machine with more than 3GB of RAM.

Here is the working command line:
```
qemu-system-arm \
-machine virt -m 3G -smp 2 -nographic \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0,id=rng-device0 \
-drive if=pflash,format=raw,unit=0,read-only=on,file=/usr/share/AAVMF/AAVMF32_CODE.fd \
-device virtio-net-pci,netdev=eth0 \
-netdev user,id=eth0,hostfwd=tcp::8022-:22 \
-drive file=autopkgtest-noble-armhf.img,if=virtio,format=qcow2
```
And here is the broken one, that ends up in a loop of `watchdog: BUG: soft lockup - CPU#1 stuck for 26s!`:
```
qemu-system-arm \
-machine virt -m 4G -smp 2 -nographic \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0,id=rng-device0 \
-drive if=pflash,format=raw,unit=0,read-only=on,file=/usr/share/AAVMF/AAVMF32_CODE.fd \
-device virtio-net-pci,netdev=eth0 \
-netdev user,id=eth0,hostfwd=tcp::8022-:22 \
-drive file=autopkgtest-noble-armhf.img,if=virtio,format=qcow2
```

Some early boot details:

3GB RAM:
```
[ 0.000000] Linux version 6.5.0-9-generic (buildd@bos02-arm64-028) (arm-linux-gnueabihf-gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.41) #9-Ubuntu SMP Fri Oct 6 23:14:49 UTC 2023 (Ubuntu 6.5.0-9.9-generic 6.5.3)
[ 0.000000] CPU: ARMv7 Processor [414fc0f0] revision 0 (ARMv7), cr=30c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] OF: fdt: Machine model: linux,dummy-virt
[ 0.000000] random: crng init done
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] efi: EFI v2.7 by Ubuntu distribution of EDK II
[ 0.000000] efi: SMBIOS=0xfffee000 SMBIOS 3.0=0xfffec000 MEMATTR=0xfe991a90 INITRD=0xfc727e90 RNG=0xfc72c010 MEMRESERVE=0xfc727d90
[ 0.000000] secureboot: Secure boot disabled
[ 0.000000] cma: Reserved 32 MiB at 0x00000000fc800000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000040000000-0x000000006fffffff]
[ 0.000000] Normal empty
[ 0.000000] HighMem [mem 0x0000000070000000-0x00000000ffffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000040000000-0x00000000ffc16fff]
[ 0.000000] node 0: [mem 0x00000000ffc17000-0x00000000ffffefff]
[ 0.000000] node 0: [mem 0x00000000fffff000-0x00000000ffffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000ffffffff]
```

4GB RAM:
```
[ 0.000000] Linux version 6.5.0-9-generic (buildd@bos02-arm64-028) (arm-linux-gnueabihf-gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.41) #9-Ubuntu SMP Fri Oct 6 23:14:49 UTC 2023 (Ubuntu 6.5.0-9.9-generic 6.5.3)
[ 0.000000] CPU: ARMv7 Processor [414fc0f0] revision 0 (ARMv7), cr=30c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] OF: fdt: Machine model: linux,dummy-virt
[ 0.000000] random: crng init done
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] efi: EFI v2.7 by Ubuntu distribution of EDK II
[ 0.000000] efi: SMBIOS=0xfffee000 SMBIOS 3.0=0xfffec000 MEMATTR=0xfe991a90 INITRD=0xfc727e90 RNG=0xfc72c010 MEMRESERVE=0xfc727d90
[ 0.000000] secureboot: Secure boot disabled
[ 0.000000] cma: Reserved 32 MiB at 0x00000000fc800000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000040000000-0x000000006fffffff]
[ 0.000000] Normal empty
[ 0.000000] HighMem [mem 0x0000000070000000-0x000000013fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000040000000-0x00000000ffc16fff]
[ 0.000000] node 0: [mem 0x00000000ffc17000-0x00000000ffffefff]
[ 0.000000] node 0: [mem 0x00000000fffff000-0x000000013fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000013fffffff]
```

Here is my version of EDK2: `qemu-efi-arm/mantic,now 2023.05-2 all [installed,automatic]`

Let me know if further details are needed.

Mate Kukri (mkukri)
Changed in edk2 (Ubuntu):
assignee: nobody → Mate Kukri (mkukri)
Skia (hyask)
description: updated
description: updated
Mate Kukri (mkukri)
Changed in edk2 (Ubuntu Mantic):
assignee: nobody → Mate Kukri (mkukri)
Revision history for this message
Mate Kukri (mkukri) wrote :

My version of qemu-system-arm doesn't even get to the kernel with >2G of RAM, the firmware just locks up.

Revision history for this message
Mate Kukri (mkukri) wrote :

Looks like that's related to the Debian version of AAVMF... Despite having the exact same version number, I managed to reproduce it with the noble qemu-efi-arm package.

Revision history for this message
Mate Kukri (mkukri) wrote :

I don't think this is an EDK2 bug anymore, it looks more like a qemu-system-arm bug. AAVMF32 + armhf kernel boots fine in qemu-system-aarch64 with 8GiB of RAM:
```
qemu-system-aarch64 \
 -machine virt \
 -m 8G \
 -drive if=pflash,format=raw,unit=0,readonly=on,file=AAVMF32_CODE.fd \
 -drive if=pflash,format=raw,unit=1,file=AAVMF32_VARS.fd \
 -drive file=autopkgtest-noble-armhf.img,if=virtio,format=qcow2
```

Mate Kukri (mkukri)
Changed in edk2 (Ubuntu Mantic):
importance: Undecided → Low
Changed in edk2 (Ubuntu Noble):
importance: Undecided → Low
summary: - Broken memory map on armhf with more than 3GB of RAM
+ armhf VMs broken with more than 3GB of RAM
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in edk2 (Ubuntu Mantic):
status: New → Confirmed
Changed in edk2 (Ubuntu):
status: New → Confirmed
Paride Legovini (paride)
Changed in qemu (Ubuntu Mantic):
status: New → Incomplete
no longer affects: qemu (Ubuntu Mantic)
no longer affects: qemu (Ubuntu Noble)
Changed in qemu (Ubuntu):
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.