systemd/247.3-3ubuntu3 ADT test failure with linux-raspi H/G/F

[Kelsey Steele]

This is a scripted bug report about ADT failures while running systemd tests for linux-raspi/5.11.0-1009.10 on hirsute. Whether this is caused by the dep8 tests of the tested source or the kernel has yet to be determined.

Testing failed on Hirsute/raspi:
    arm64: https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/arm64/s/systemd/20210528_100011_6f6f0@/log.gz

Testing failed on Groovy/raspi:
    arm64: https://autopkgtest.ubuntu.com/results/autopkgtest-groovy/groovy/arm64/s/systemd/20210528_084522_9b779@/log.gz

Testing failed on Focal/raspi:
    arm64: https://autopkgtest.ubuntu.com/results/autopkgtest-focal/focal/arm64/s/systemd/20210528_092727_7c33b@/log.gz

[Kleber Sacilotto de Souza (kleber-souza)]

This seems to be a testbed issue and not a linux-raspi kernel bug.

Looking at the logs, the first reboot with the latest raspi kernel installed is successful. However, some subsequent reboot will fail. Looking at the openstack nova console logs, we can see that the testbed is running the generic arm64 kernel:

------- nova console-log 225fbcdf-170f-4f77-bf46-1435821e0776 (adt-hirsute-arm64-systemd-20210528-035426) ------
[2J[01;01H[=3h[2J[01;01H[2J[01;01H[=3h[2J[01;01H[2J[01;01H[=3h[2J[01;01H[0m[35m[40m[0m[37m[40merror: no suitable video mode found.
EFI stub: Booting Linux Kernel...
EFI stub: EFI_RNG_PROTOCOL unavailable, KASLR will be disabled
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x503f0002]
[ 0.000000] Linux version 5.11.0-17-generic (buildd@bos02-arm64-057) (gcc (Ubuntu 10.3.0-1ubuntu1) 10.3.0, GNU ld (GNU Binutils for Ubuntu) 2.36.1) #18-Ubuntu SMP Thu May 6 17:29:33 UTC 2021 (Ubuntu 5.11.0-17.18-generic 5.11.12)

When the system is rebooted, it fails to come back and hangs at:

[ 145.624340] reboot: Restarting system
[2J[01;01H[=3h[2J[01;01H[2J[01;01H[=3h[2J[01;01H[2J[01;01H[=3h[2J[01;01H[0m[35m[40m[0m[37m[40merror: no suitable video mode found.
EFI stub: Booting Linux Kernel...
EFI stub: EFI_RNG_PROTOCOL unavailable, KASLR will be disabled
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...

[Juerg Haefliger (juergh)]

This is a general problem when running any raspi kernel in ADT since the raspi kernels don't support ACPI. Software configuration seems to be different across different arm64 host HW in the ADT OpenStack cloud. Older HW has older qemu-efi installed which presents both ACPI *and* a DT to the guest and the raspi kernel boots fine in VMs running on such host HW. Newer host HW has newer qemu-efi installed which only presents ACPI and VMs running on such host HW fail to boot with the raspi kernel, leading to the above error/timeout.

The solution is to support a special ADT trigger 'qemu-eif-noacpi/0' which calls a script during testbed setup that injects a startup.nsh script into the EFI partition of the VM and changes NextBoot to the EFI shell. The VM then reboots after testbed setup, executes startup.nsh which sets a hidden EFI variable to instruct QEMU to present a DT and finally resets the VM for this change to take effect. When the VM comes back, the guest OS is presented with a DT and the raspi kernel boots and the test continues.

That's the theory. However, the reboot method after testbed setup was recently changed for some reason. It now uses 'openstack reboot --hard' which results in a libvirt domain destroy and recreate which in turn results in loosing the EFI variable setting so the VM comes back with ACPI again and no DT. And the raspi kernel no longer boots...

[Juerg Haefliger (juergh)]

Time to turn off ADT testing for raspi kernels since it adds no real value.

[Juerg Haefliger (juergh)]

Clarification: The raspi kernel contains a bunch of downstream HW specific patches pulled primarily from the raspberrypi kernel. These code modifications are never exercised in an OpenStack VM or LXD container so it's rather pointless to do so.