[Ubuntu 1810] Kdump fails to dump vmcore and enters initramfs inside Power9 KVM guest
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Fix Released
|
High
|
Canonical Kernel Team | ||
makedumpfile (Ubuntu) |
Fix Released
|
High
|
Canonical Kernel Team |
Bug Description
Kdump fails to dump vmcore even with workaround suggested
This issue is submitted to
track on Power9 Guest where it uses file type qcow2 disk (virtio-scsi)
Boot Log: (Attached full console log)
[ 3.754031] 32regs : 19616.000 MB/sec
[ 3.794031] 32regs_prefetch: 17280.000 MB/sec
[ 3.834030] altivec : 22480.000 MB/sec
[ 3.834063] xor: using function: altivec (22480.000 MB/sec)
done.
Begin: Running /scripts/
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/
Begin: Waiting for root file system ... Begin: Running /scripts/
done.
mdadm: No devices listed in conf file were found.
mdadm: No devices listed in conf file were found.
mdadm: No devices listed in conf file were found.
done.
Gave up waiting for root file system device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Missing modules (cat /proc/modules; ls /dev)
ALERT! UUID=5e1fe9e9-
BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu4) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs)
Contact Information = Balamuruhan S / <email address hidden>
---uname output---
Guest Kernel: 4.18.0-11-generic
Host Kernel: 4.18.0-11-generic
Machine Type = Boston
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1. Have a healthy KVM guest with Ubuntu 1810 with kernel 4.18.0-11-generic
2. Install kdump, kexec and crash tools in the guest,
# dpkg -l | grep crash
ii apport 2.20.10-0ubuntu13.1 all automatically generate crash reports for debugging
ii crash 7.2.3+real-1 ppc64el kernel debugging utility, allowing gdb like syntax
ii kdump-tools 1:1.6.4-2ubuntu1 ppc64el scripts and tools for automating kdump (Linux crash dumps)
ii linux-crashdump 4.18.0.11.12 ppc64el Linux kernel crashdump setup for the latest generic kernel
ii python3-apport 2.20.10-0ubuntu13.1 all Python 3 library for Apport crash report handling
3. Ensure workaround suggested in Bug 172389 is followed by uncomment the `KDUMP_
and change nr_cpus to maxcpus in /etc/default/
# cat /etc/default/
# KDUMP_CMDLINE - The default is to use the contents of /proc/cmdline.
# Set this variable to override /proc/cmdline.
# KDUMP_CMDLINE_
#KDUMP_CMDLINE=""
KDUMP_CMDLINE_
4. restart the kdump tools service,
# service kdump-tools restart
# service kdump-tools status
? kdump-tools.service - Kernel crash dump capture service
Loaded: loaded (/lib/systemd/
Active: active (exited) since Mon 2018-12-03 02:34:03 CST; 1 weeks 3 days ago
Process: 1560 ExecStart=
Main PID: 1560 (code=exited, status=0/SUCCESS)
Dec 03 02:34:02 ubuntu1810 systemd[1]: Starting Kernel crash dump capture servic
Dec 03 02:34:02 ubuntu1810 kdump-tools[1560]: Starting kdump-tools: * Creating
Dec 03 02:34:02 ubuntu1810 kdump-tools[1560]: * Creating symlink /var/lib/kdump
Dec 03 02:34:03 ubuntu1810 kdump-tools[1560]: Modified cmdline:
Dec 03 02:34:03 ubuntu1810 kdump-tools[1560]: * loaded kdump kernel
Dec 03 02:34:03 ubuntu1810 kdump-tools[1678]: /sbin/kexec -p --command-line="BOO
Dec 03 02:34:03 ubuntu1810 kdump-tools[1679]: loaded kdump kernel
Dec 03 02:34:03 ubuntu1810 systemd[1]: Started Kernel crash dump capture service
5. check kdump-config is state is ready to dump,
# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.
KDUMP_COREDIR: /var/crash
crashkernel addr:
/var/
kdump initrd:
/var/
current state: ready to kdump
6. Reboot the guest and check for kernel cmdline whether crashkernel is included,
# cat /proc/cmdline
BOOT_IMAGE=
7. enable sysrq and trigger the crash,
# echo 1 > /proc/sys/
# cat /proc/sys/
1
# echo c > /proc/sysrq-trigger
kdump fails to generate vmcore after crash instead reboots and enters initramfs
Attachment:
1. Guest console log
2. Guest sosreport
3. Guest xml
tags: | added: architecture-ppc64le bugnameltc-174078 severity-high targetmilestone-inin--- |
Changed in ubuntu: | |
assignee: | nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) |
affects: | ubuntu → linux (Ubuntu) |
affects: | linux (Ubuntu) → makedumpfile (Ubuntu) |
Changed in ubuntu-power-systems: | |
status: | New → Triaged |
importance: | Undecided → High |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
Changed in makedumpfile (Ubuntu): | |
importance: | Undecided → High |
status: | New → Triaged |
assignee: | Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Kernel Team (canonical-kernel-team) |
tags: | added: kernel-da-key |
Changed in ubuntu-power-systems: | |
status: | Triaged → Incomplete |
Changed in ubuntu-power-systems: | |
status: | Incomplete → Triaged |
------- Comment From <email address hidden> 2019-01-18 00:59 EDT-------
Ubuntu, please provide update