cloud images slow to boot under kvm

[Ross Vandegrift]

The Ubuntu cloud images hang for 15-30 seconds when run in qemu w/kvm. This happens before Grub's welcome message, so I think it's happening before the bootloader. This does not happen with other OS images using the exact same setup.

- With BIOS boot, it sits at the "Booting from Hard Disk..." screen.
- With EFI boot, it sits at "BdsDxe: starting Boot0001 "UEFI QEMU HARDDISK QM00001" from ... (pci details)"

Both 22.04 and 23.10 behave the same. I'm using qemu 7.2 from debian bookworm on linux 6.5.10 from bookworm-backports.

A recent Debian bookworm and Fedora image (exact urls below) both reach the bootloader in less than a second from the above points. So this seems related to the ubuntu images.

It'd be nice to speed this up, though so far I don't have any idea for the cause.

Thanks,
Ross

22.04 - https://cloud-images.ubuntu.com/releases/22.04/release-20240301/ubuntu-22.04-server-cloudimg-amd64.img
23.10 - https://cloud-images.ubuntu.com/releases/23.10/release-20240301/ubuntu-23.10-server-cloudimg-amd64.img
bookworm - http://cloud.debian.org/images/cloud/bookworm/20240211-1654/debian-12-genericcloud-amd64-20240211-1654.qcow2
fedora - https://download.fedoraproject.org/pub/fedora/linux/releases/39/Cloud/x86_64/images/Fedora-Cloud-Base-39-1.5.x86_64.qcow2

[Christian Ehrhardt  (paelzer)]

Hi Ross,
thanks for the report, and that is odd indeed.
It sounds like the initialization needing a fallback of some sort to then go to the next step and get to the bootloader.

I was quickly retrying and did not see/reproduce that slowdown in the bootloader.
It passes that stage rather quick, let us say ~1 second.

I didn't care about the later system to work (no PW setting or such), just a cmdline to see how fast it would boot (initialize, boot-load, kernel, complete system boot), the command I ran was:

$ sudo qemu-system-x86_64 -m 1G -cpu host -smp 2 -machine accel=kvm,type=q35 -drive file=/tmp/ubuntu-22.04-server-cloudimg-amd64.img,if=virtio,format=qcow2 -nographic

I tried the above 1:6.2+dfsg-2ubuntu6.17 in Ubuntu Jammy, as well 1:8.2.2+ds-1 in Debian sid and 1:7.2+dfsg-7+deb12u5 in Debian bookworm. They are all fast with the 22.04 image you referred to.

I used that commandline to - for now - to avoid having to look for well meant distracting magic of libvirt or other higher level things. I see nothing super special in my commandline nor can I see the issue.
Hence I need to ask you which command or tooling config did you use to start the guest when you see the difference in boot speed?

[Ross Vandegrift (ross-kallisti)]

Hi Christian,
Sorry for not including my qemu command line. Thanks for including yours, that lets me spot the difference.

I've been using `-machine accel=kvm` (so the default, pc-i440fx). Switching to q35 fixes the issue, and the other OS images are still fast.

Kind of perplexing that it'd only affect Ubuntu - but I'm happy to have a workaround.

Thanks,
Ross

[Sergio Durigan Junior (sergiodj)]

Hi Ross,

Thanks for the feedback. It's a strange situation, indeed. And I have to say that I cannot reproduce the issue here even after forcing qemu to use pc-i440fx-{mantic,jammy}, which is what happens when you don't specify type=q35. The VM boots without issues. Of course, the first time it boots it takes longer than usual because of the snap seed population and other first-boot tasks that are executed, but otherwise everything else happens as expected.

I am going to keep this bug as Incomplete for now because we're unable to reproduce the issue on our end. Please let us know if you have more information that could help us trigger the problem.

Thanks.

[Christian Ehrhardt  (paelzer)]

Ack to everything Sergio said, but even when trying with the qemu 7.2 in Debian that you have.
No huge slowdown in the transition from early boot stages to seeing the kernel load (again ~1 second).