Comment 22 for bug 1970672

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

### VERIFICATION-FAILED FOCAL-PROPOSED ###

makedumpfile original version: 1:1.6.7-1ubuntu2.4
makedumpfile proposed version: 1:1.6.7-1ubuntu2.5
kernel version: 5.15.0-89-generic hwe

sudo apt install -y linux-crashdump

sudo vim /etc/default/grub.d/kdump-tools.cfg
 # Add the line at the top of the file below USE_KDUMP=1
 LOAD_KEXEC=true
 # Uncomment the makedumpfile line and change 31 to 32
 MAKEDUMP_ARGS="-c -d 32"
 exit vim

sudo vim /etc/default/grub.d/kdump-tools.cfg
change 192 to either 256M or 512M

sudo sysctl -w kernel.sysrq=1

sudo apt-get update-grub
sudo reboot

sudo su
kdump-config show

Needs to look similar to:
root@focal:/home/ubuntu# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x73000000
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.15.0-89-generic
kdump initrd:
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-5.15.0-89-generic
current state: ready to kdump

kexec command:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic root=UUID=975b9a95-b58e-48da-bd23-dd01b13bcbad ro quiet splash vt.handoff=7 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

*most importantly the addr is not zero and the state is: ready to kdump.

# trigger crash

echo c > /proc/sysrq-trigger

... wait for reboot and login

cd /var/crash/

There is a folder with a datetimestamp of the crash, inside the folder is the vmcore and dmesg files
There is also a file called linux-image*.crash in /var/crash/

*Note: In bionic we get the cp message

[ 6.946187] kdump-tools[622]: Starting kdump-tools: * running makedumpfile -c -d 32 /proc/vmcore /var/crash/202311221422/dump-incomplete
[ 6.959932] kdump-tools[622]: Dump_level(32) is invalid.
[ 6.964316] kdump-tools[622]: makedumpfile Failed.
[ 6.976231] kdump-tools[622]: * kdump-tools: makedumpfile failed, falling back to 'cp'
[ 25.084729] kdump-tools[622]: * kdump-tools: saved vmcore in /var/crash/202311221422
[ 26.355039] kdump-tools[622]: * running makedumpfile --dump-dmesg /proc/vmcore /var/crash/202311221422/dmesg.202311221422
[ 26.436513] kdump-tools[622]: The dmesg log is saved to /var/crash/202311221422/dmesg.202311221422.
[ 26.443208] kdump-tools[622]: makedumpfile Completed.
[ 26.449066] kdump-tools[622]: * kdump-tools: saved dmesg content in /var/crash/202311221422
[ 26.463950] kdump-tools[622]: Wed, 22 Nov 2023 14:22:50 +0000
[ 26.490318] kdump-tools[622]: Rebooting.

*Note: In Focal we don't

Nov 28 13:35:54 focal kdump-tools[604]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: n: failed to create symbolic link '/va: No such file or directory
Nov 28 13:35:54 focal kdump-tools[713]: kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.0-89-generic
Nov 28 13:35:57 focal kdump-tools[667]: * Creating symlink /var/lib/kdump/initrd.img
Nov 28 13:35:57 focal kdump-tools[667]: * Invalid symlink : /var/lib/kdump/vmlinuz
Nov 28 13:35:57 focal kdump-tools[667]: * Creating symlink /var/lib/kdump/vmlinuz
Nov 28 13:35:58 focal kdump-tools[667]: * loaded kdump kernel
Nov 28 13:35:58 focal kdump-tools[3337]: /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic root=UUID=975b9a95-b58e-48da-bd23-dd01b13bcbad ro quiet splash vt.handoff=7 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
Nov 28 13:35:58 focal kdump-tools[3338]: loaded kdump kernel

-------

enabled proposed repo

Remove any crashes from /var/crash/

install proposed version of makedumpfile

"follow similar steps above if on new machine"

/var/log/syslog
Nov 28 15:10:19 focal kdump-tools[674]: Starting kdump-tools:
Nov 28 15:10:19 focal kdump-tools[674]: * Creating symlink / * Creating symlink /var/lib/kdump/initrd.img
Nov 28 15:10:20 focal kdump-tools[674]: * loaded kdump kernel
Nov 28 15:10:20 focal kdump-tools: /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic root=UUID=1887d6a1-cdd3-474f-a3a8-53703a658358 ro quiet splash vt.handoff=7 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
Nov 28 15:10:20 focal kdump-tools: loaded kdump kernel

You can produce a vmcore 2 ways.
1. by editing the file /etc/default/kdump-tools to
commenting in MAKEDUMPFILE_ARGS="-c -d 32"

2. by command
mv /usr/bin/makedumpfile /usr/bin/makedumpfile.bak

My results show that with no changes to cause a crash, we see a dmesg and dump files created in the /var/crash/timestamp folder which is the correct workflow. However, purposely causing a crash to happen results in a vmcore the size of the installed RAM.

Cheers,
Heather Lemon

*If this needs to be re-tested, please make a note and explanation.

### VERIFICATION-FAILED FOCAL-PROPOSED ###