Comment 5 for bug 1948862

Revision history for this message
Matthew Ruffell (mruffell) wrote :

Performing verification for Bionic.

I installed the current 4.15.0-162-generic kernel from -updates to a host in segmaas, and brought up my Bionic VM I installed for the reproducer. The Bionic VM has linux-crashdump configured, with nr_cpus configured to 8:

ubuntu@bionic-vm:~$ kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-4.15.0-159-generic
kdump initrd:
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.15.0-159-generic
current state: ready to kdump

kexec command:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-4.15.0-159-generic root=UUID=5c23b432-4d00-4a01-8ecd-964803c8e10a ro console=tty1 console=ttyS0 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=8 irqpoll nousb ata_piix.prefer_ms_hyperv=0 ignore_loglevel nmi_debug=state,regs apic_extnmi=none" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

I then triggered the crashdump in the VM:

$ sudo -s
# echo c > /proc/sysrq-trigger

On the host, we look at the tail of:

$ sudo tail -f /var/log/libvirt/qemu/bionic-vm.log
...
KVM internal error. Suberror: 1
emulation failure
EAX=0000de8f EBX=00000000 ECX=0000008f EDX=00000600
ESI=00000000 EDI=00000000 EBP=00000000 ESP=0000f90c
EIP=0000cdb1 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 00009b00
SS =de00 000de000 0000ffff 00009300
DS =de00 000de000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=32b2a000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=66 83 c4 28 66 5b 66 c3 66 56 66 53 66 52 b1 8f 88 c8 e6 70 <e4> 71 66 0f b6 f0 66 89 f2 67 88 54 24 03 88 c8 e6 70 66 31 db 88 d8 e6 71 66 56 66 68 1a

$ virsh list
 Id Name State
----------------------------------------------------
 1 bionic-vm paused

Okay, we can reproduce the issue.

I then enabled -proposed, and installed the 4.15.0-163-generic kernel:

$ uname -rv
4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021

I started the same VM, verified it has a crashkernel with nr_cpus=8:

$ kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-4.15.0-159-generic
kdump initrd:
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.15.0-159-generic
current state: ready to kdump

kexec command:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-4.15.0-159-generic root=UUID=5c23b432-4d00-4a01-8ecd-964803c8e10a ro console=tty1 console=ttyS0 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=8 irqpoll nousb ata_piix.prefer_ms_hyperv=0 ignore_loglevel nmi_debug=state,regs apic_extnmi=none" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

I then triggered the crashdump:

$ sudo -s
# echo c > /proc/sysrq-trigger

On the host:

$ sudo tail -f /var/log/libvirt/qemu/bionic-vm.log
...
2021-11-11T02:41:45.970264Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
2021-11-11T02:41:45.971185Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]

$ virsh list
 Id Name State
----------------------------------------------------
 1 bionic-vm running

Checking the VM, the crashdump completed successfully and restarted correctly.

$ virsh console bionic-vm
...
Copying data : [100.0 %] \ eta: 0s
[ 12.034361] kdump-tools[874]: The dumpfile is saved to /var/crash/202111110246/dump-incomplete.
[ 12.036787] kdump-tools[874]: makedumpfile Completed.
[ 12.038752] kdump-tools[874]: * kdump-tools: saved vmcore in /var/crash/202111110246
[ 12.058489] kdump-tools[874]: * running makedumpfile --dump-dmesg /proc/vmcore /var/crash/202111110246/dmesg.202111110246
[ 12.069700] kdump-tools[874]: The dmesg log is saved to /var/crash/202111110246/dmesg.202111110246.
[ 12.072574] kdump-tools[874]: makedumpfile Completed.
[ 12.074245] kdump-tools[874]: * kdump-tools: saved dmesg content in /var/crash/202111110246
[ 12.159319] kdump-tools[874]: Thu, 11 Nov 2021 02:46:54 +0000
...

I then did this a further 4 times, each time the crashdump completed succesfully.
I then destroyed and started the VM fresh, and kept crashdumping. It worked every time.

The 4.15.0-163-generic kernel in -proposed fixes the issue. Happy to mark as verified.