set reasonable default cmdlines for arm64

Bug #1763532 reported by dann frazier
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kexec-tools (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
dann frazier
Artful
Invalid
Undecided
Unassigned
makedumpfile (Debian)
Fix Released
Unknown
makedumpfile (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
dann frazier
Artful
Fix Released
Undecided
dann frazier

Bug Description

[Impact]
Crash dump support doesn't work by default on any known arm64 platform. Users have to figure out working incantations for the crashkernel= parameter and the parameters passed to the crash-dump kernel themselves and manual edit 2 different config files.

*NOTE* one of these config files moved from kexec-tools to kdump-tools between xenial & artful - that's why kexec-tools is only updated for xenial.

*NOTE* the debdiffs here are large because a few files have been renamed to "*.in" so they can be used as templates for generating per-architecture files at build time.

[Test Case]
Using either a Centriq 2400 or a Cavium ThunderX CRB (these platforms are known to support crashdump), and with a kernel >= 4.13 (xenial-ga did not support crashdump on arm64):

sudo apt install kdump-tools
(reboot, so crashkernel= is added to the kernel commandline)
echo c | sudo tee /proc/sysrq-trigger

Without having to edit any config files, crash dump should occur, with artifacts collected in /var/crash.

If you want to verify that the dump is usable, install the corresponding linux-image-<ver>-dbgsym package and run:

sudo crash /usr/lib/debug/boot/vmlinux-<ver> /var/crash/<crash>/dump.<crash>

crash should successfully load, placing you at a "crash>" prompt. At that prompt, you can issue the 'bt' command to see a backtrace.
Note: you need crash from zesty (7.1.8-1ubuntu1) or later.

[Regression Risk]
The biggest regression risk I see here is that we're introducing build-time mechanisms for generating config files per-architecture, and bugs in that implementation could lead to incorrect settings for non-arm64 architectures. To validate that, I've staged these update in a PPA (ppa:dannf/arm64-kdump), and I extracted those debs in a directory next to the current debs and diff'd the two trees[1]. Ignoring binary files, diff shows that only config files in the arm64 build are different, as intended. Note that, while this config file generation also exists in bionic, I modified it for these backports so that it retains the parameter order from the previous xenial/artful builds. This is so that the config files for non-arm64 architectures will be identical to the previous builds, and dpkg won't unnecessarily prompt admins to resolve conflicts if they've customized them.

[1]
$ for arch in amd64 arm64 armhf i386 ppc64el s390x; do
    diff -u0 -rpN ${arch}-cur ${arch}-new | grep -v ^Binary
  done

Changed in makedumpfile (Debian):
status: Unknown → Fix Released
dann frazier (dannf)
Changed in kexec-tools (Ubuntu Artful):
status: New → Invalid
Changed in makedumpfile (Ubuntu Artful):
status: New → In Progress
Changed in makedumpfile (Ubuntu Xenial):
status: New → In Progress
Changed in makedumpfile (Ubuntu):
status: New → Fix Released
Changed in kexec-tools (Ubuntu):
status: New → Fix Released
Changed in kexec-tools (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → dann frazier (dannf)
Changed in makedumpfile (Ubuntu Xenial):
assignee: nobody → dann frazier (dannf)
Changed in makedumpfile (Ubuntu Artful):
assignee: nobody → dann frazier (dannf)
dann frazier (dannf)
description: updated
dann frazier (dannf)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello dann, or anyone else affected,

Accepted makedumpfile into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.1-2ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in makedumpfile (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello dann, or anyone else affected,

Accepted makedumpfile into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.5.9-5ubuntu0.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in makedumpfile (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello dann, or anyone else affected,

Accepted kexec-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/kexec-tools/1:2.0.10-1ubuntu2.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in kexec-tools (Ubuntu Xenial):
status: In Progress → Fix Committed
Revision history for this message
dann frazier (dannf) wrote :

For xenial and artful, I have verified by:

1) Deploying an arm64 system (Cavium ThunderX CRB1S), enabling proposed, installing kdump-tools, rebooting, then triggering a dump.
2) Regression/upgrade tested on x86 by launching a guest, installing kdump-tools, rebooting, then trigger a dump. Next, enabled proposed, dist-upgraded, rebooted, then triggering another dump.

Everything worked as expected.

tags: added: verification-done verification-done-artful verification-done-xenial
removed: verification-needed verification-needed-artful verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.1-2ubuntu0.2

---------------
makedumpfile (1:1.6.1-2ubuntu0.2) artful; urgency=medium

  * kdump-tools: Add reasonable default 'crashkernel' parameter and kdump
    kernel cmdline for arm64. The files that contain the default kernel
    cmdline are now generated per-architecture at build-time. (LP: #1763532).

 -- dann frazier <email address hidden> Fri, 13 Apr 2018 17:49:36 -0600

Changed in makedumpfile (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for makedumpfile has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.5.9-5ubuntu0.7

---------------
makedumpfile (1:1.5.9-5ubuntu0.7) xenial; urgency=medium

  * kdump-tools: Add reasonable default kdump kernel cmdline for arm64.
    The files that contain the default kernel cmdline are now generated
    per-architecture at build-time. Consequently, kdump-tools is no
    longer Architecture: all. (LP: #1763532).

 -- dann frazier <email address hidden> Fri, 13 Apr 2018 17:50:04 -0600

Changed in makedumpfile (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.10-1ubuntu2.5

---------------
kexec-tools (1:2.0.10-1ubuntu2.5) xenial; urgency=medium

  * The default crashkernel parameter didn't work on any known arm64
    systems, set a reasonable default instead (LP: #1763532).

 -- dann frazier <email address hidden> Fri, 13 Apr 2018 17:50:27 -0600

Changed in kexec-tools (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.