Make reset_devices parameter default for kdump

Bug #1800566 reported by Guilherme G. Piccoli
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
makedumpfile (Ubuntu)
Fix Released
High
Guilherme G. Piccoli
Trusty
Won't Fix
Undecided
Unassigned
Xenial
Fix Released
High
Guilherme G. Piccoli
Bionic
Fix Released
High
Guilherme G. Piccoli
Cosmic
Won't Fix
Undecided
Unassigned
Disco
Won't Fix
Undecided
Unassigned
Eoan
Fix Released
High
Guilherme G. Piccoli
Focal
Fix Released
High
Guilherme G. Piccoli

Bug Description

[Impact]

* Kdump does not configure by default the crash kernel to perform a device reset by default, by passing the "reset_devices" parameter.

* Kernel has the "reset_devices" parameter that drivers can opt-in, and perform special activity in case this parameter is parsed from command-line. For example, in kdump kernels it hints the drivers that they are booting from a non-healthy condition and needs to issue some form of reset to the adapter, like clearing DMA mapping in their firmware for example. Users currently (kernel v5.5-rc2) are: aacraid, hpsa, ipr, megaraid_sas, mpt3sas, smartpqi, xenbus.

This should be enabled by default in the kdump config file to be added in the kdump kernel command-line for all versions.

[Test Case]

1) Deploy a Bionic VM e.g. with uvt-kvm
2) Install the kdump-tools package
3) Run `kdump-config test`and check for the 'reset_devices' parameter:

$ kdump-config test
...
kexec command to be used:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-4.15.0-45-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0 nr_cpus=1 systemd.unit=kdump-tools.service irqpoll nousb ata_piix.prefer_ms_hyperv=0" /var/lib/kdump/vmlinuz

[Regression Potential]

The regression potential is low, since it doesn't need any changes in makedumpfile code and we're only adding a parameter on the crash kernel command-line. The risks are related with bad behavior with the kernel when using "reset_devices", like if the driver has bugs in this path. It's considered safer to have the option (and this way prevent problems for booting a unhealthy kernel with potential stuck DMAs in the devices) than not having it.

Changed in makedumpfile (Ubuntu Cosmic):
importance: Undecided → High
Changed in makedumpfile (Ubuntu Bionic):
importance: Undecided → High
Changed in makedumpfile (Ubuntu Xenial):
importance: Undecided → High
Changed in makedumpfile (Ubuntu Trusty):
importance: Undecided → High
assignee: nobody → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Trusty):
assignee: Guilherme G. Piccoli (gpiccoli) → Heitor R. Alves de Siqueira (halves)
Changed in makedumpfile (Ubuntu Xenial):
assignee: nobody → Heitor R. Alves de Siqueira (halves)
Changed in makedumpfile (Ubuntu Bionic):
assignee: nobody → Heitor R. Alves de Siqueira (halves)
Changed in makedumpfile (Ubuntu Cosmic):
assignee: nobody → Heitor R. Alves de Siqueira (halves)
Changed in makedumpfile (Ubuntu Disco):
assignee: Mauricio Faria de Oliveira (mfo) → Heitor R. Alves de Siqueira (halves)
Changed in makedumpfile (Ubuntu Cosmic):
status: New → Confirmed
Changed in makedumpfile (Ubuntu Bionic):
status: New → Confirmed
Changed in makedumpfile (Ubuntu Xenial):
status: New → Confirmed
Changed in makedumpfile (Ubuntu Trusty):
status: New → Confirmed
description: updated
Revision history for this message
Heitor Alves de Siqueira (halves) wrote :

Debdiffs for this bug were attached to LP#1800562

description: updated
tags: removed: sts
tags: added: seg
Changed in makedumpfile (Ubuntu):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Xenial):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Bionic):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Disco):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Trusty):
status: Confirmed → Won't Fix
Changed in makedumpfile (Ubuntu Cosmic):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Trusty):
assignee: Heitor Alves de Siqueira (halves) → Guilherme G. Piccoli (gpiccoli)
summary: - Make the reset_devices parameter default for kdump kernels
+ Make reset_devices parameter default for kdump and decouple kdump
+ systemd service from the KDUMP_CMDLINE_APPEND
description: updated
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Re: Make reset_devices parameter default for kdump and decouple kdump systemd service from the KDUMP_CMDLINE_APPEND

This is the debdiff with this LP's proposed modifications.
I'd like to specially thanks Cascardo and Heitor for the discussions and joint work in this issue.

Changed in makedumpfile (Ubuntu Eoan):
status: Confirmed → In Progress
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "lp1800566_eoan.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Mathew Hodson (mhodson)
Changed in makedumpfile (Ubuntu Cosmic):
status: Confirmed → Won't Fix
Changed in makedumpfile (Ubuntu Eoan):
status: In Progress → Confirmed
Changed in makedumpfile (Ubuntu Focal):
status: In Progress → Confirmed
Changed in makedumpfile (Ubuntu Disco):
status: Confirmed → Won't Fix
Changed in makedumpfile (Ubuntu Focal):
status: Confirmed → In Progress
Changed in makedumpfile (Ubuntu Eoan):
status: Confirmed → In Progress
Changed in makedumpfile (Ubuntu Bionic):
status: Confirmed → In Progress
Changed in makedumpfile (Ubuntu Xenial):
status: Confirmed → In Progress
Dan Streetman (ddstreet)
Changed in makedumpfile (Ubuntu Disco):
status: Won't Fix → In Progress
summary: - Make reset_devices parameter default for kdump and decouple kdump
- systemd service from the KDUMP_CMDLINE_APPEND
+ Make reset_devices parameter default for kdump
description: updated
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

After some attempt to merge the work needed in LP #1816743 here, we decided to split the bugs and only work the 'reset_devices' addition here.

Cheers,

Guilherme

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.6-4ubuntu1

---------------
makedumpfile (1:1.6.6-4ubuntu1) focal; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Merge from Debian unstable. Remaining changes:
    - Bump amd64 crashkernel from 384M-:128M to 512M-:192M.
  * Use reset_devices as a cmdline parameter. (LP: #1800566)
  * Use kdump-config reload after cpu or memory hotplug. (LP: #1828596)

  [ Guilherme G. Piccoli ]
  * Add a systemd-resolved service dependency in order kdump-tools is able
    to resolve DNS when in kdump boot. (LP: #1856323)

makedumpfile (1:1.6.6-4) unstable; urgency=medium

  * Let the kernel decide the crashkernel offset for ppc64el (LP: #1741860)
  * kdump-config: implement try-reload
  * udev: hotplug: use try-reload
  * Set Rules-Requires-Root to no

makedumpfile (1:1.6.6-3) unstable; urgency=medium

  * Add a reload command.
  * Use kdump-config reload after cpu or memory hotplug.
  * Use reset_devices as a cmdline parameter.

 -- Thadeu Lima de Souza Cascardo <email address hidden> Wed, 18 Dec 2019 14:38:51 -0300

Changed in makedumpfile (Ubuntu Focal):
status: In Progress → Fix Released
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

For this LP SRU submission, the following candidate packages were tested in amd64 arch:
* Xenial, candidate version 1.6.3-2~16.04.2;
* Bionic, candidate version 1.6.5-1ubuntu1~18.04.4;
* Disco, candidate version 1.6.5-1ubuntu1.4;
* Eoan, candidate version 1.6.6-2ubuntu2;

The test consisted in installing the package and check "kdump-config show" output to validate if the "reset_devices" parameter was added to kdump command-line.

Cheers,

Guilherme

Changed in makedumpfile (Ubuntu Disco):
status: In Progress → Won't Fix
importance: High → Undecided
assignee: Guilherme G. Piccoli (gpiccoli) → nobody
Changed in makedumpfile (Ubuntu Cosmic):
importance: High → Undecided
assignee: Guilherme G. Piccoli (gpiccoli) → nobody
Revision history for this message
Andy Whitcroft (apw) wrote : Please test proposed package

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.6-2ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in makedumpfile (Ubuntu Eoan):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-eoan
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (makedumpfile/1:1.6.6-2ubuntu2)

All autopkgtests for the newly accepted makedumpfile (1:1.6.6-2ubuntu2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.6-2ubuntu2 (i386, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Andy Whitcroft (apw) wrote : Please test proposed package

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.5-1ubuntu1~18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in makedumpfile (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (makedumpfile/1:1.6.5-1ubuntu1~18.04.4)

All autopkgtests for the newly accepted makedumpfile (1:1.6.5-1ubuntu1~18.04.4) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.5-1ubuntu1~18.04.4 (ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Andy Whitcroft (apw) wrote : Please test proposed package

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.3-2~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in makedumpfile (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

I've verified this LP in all 3 releases, by installing the package sin -proposed and checking the command-line of kdump kernel, which contains "reset_devices" for all the 3 versions tested:

1:1.6.3-2~16.04.2 for xenial,
1:1.6.5-1ubuntu1~18.04.4 for bionic,
1:1.6.6-2ubuntu2 for eoan.

Cheers,

Guilherme

Changed in makedumpfile (Ubuntu Trusty):
importance: High → Undecided
assignee: Guilherme G. Piccoli (gpiccoli) → nobody
tags: added: verification-done verification-done-bionic verification-done-eoan verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-eoan verification-needed-xenial
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

Sorry for the typo in last comment: "package sin - proposed" -> "packages in -proposed" !

Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for makedumpfile has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.6-2ubuntu2

---------------
makedumpfile (1:1.6.6-2ubuntu2) eoan; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Fixes for DLPAR cpu add operation (LP: #1828596)
    - d/kdump-config.in: Add a reload command.
    - d/kdump-config.in: implement try-reload.
    - d/50-kdump-tools.rules: Use kdump-config reload after cpu or memory hotplug
    - d/50-kdump-tools.rules: use try-reload instead.
  * d/rules: Use reset_devices as a cmdline parameter. (LP: #1800566)

  [ Guilherme G. Piccoli ]
  * d/kdump-tools-dump.service: Add a systemd-resolved service dependency
    in order to make kdump-tool able to resolve DNS when in kdump boot.
    (LP: #1856323)
  * d/p/0003-Increase-SECTION_MAP_LAST_BIT-to-4.patch: x86_64: Fix an error due
    to makedumpfile being out-of-sync with recent kernels. (LP: #1857616)

 -- <email address hidden> (Guilherme G. Piccoli) Fri, 03 Jan 2020 16:10:19 -0300

Changed in makedumpfile (Ubuntu Eoan):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.5-1ubuntu1~18.04.4

---------------
makedumpfile (1:1.6.5-1ubuntu1~18.04.4) bionic; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Fixes for DLPAR cpu add operation (LP: #1828596)
    - d/kdump-config.in: Add a reload command.
    - d/kdump-config.in: implement try-reload.
    - d/50-kdump-tools.rules: Use kdump-config reload after cpu or memory hotplug
    - d/50-kdump-tools.rules: use try-reload instead.
  * d/rules: Use reset_devices as a cmdline parameter. (LP: #1800566)

  [ Guilherme G. Piccoli ]
  * d/kdump-tools-dump.service: Add a systemd-resolved service dependency
    in order to make kdump-tool able to resolve DNS when in kdump boot.
    (LP: #1856323)
  * Fix an error due to makedumpfile being out-of-sync with recent kernels.
    (LP: #1857616)
    - d/p/0004-x86_64-fix-get_kaslr_offset_x86_64-to-return-kaslr_offset-correctly.patch
    - d/p/0005-Increase-SECTION_MAP_LAST_BIT-to-4.patch

 -- <email address hidden> (Guilherme G. Piccoli) Fri, 03 Jan 2020 13:14:39 -0300

Changed in makedumpfile (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.3-2~16.04.2

---------------
makedumpfile (1:1.6.3-2~16.04.2) xenial; urgency=medium

  * Let the kernel decide the crashkernel offset for ppc64el (LP: #1741860)
  * Reload kdump after memory/CPU hotplug. (LP: #1655280)
  * Use a different service for vmcore dump. (LP: #1811692)
  * Reload kdump when CPU is brought online. (LP: #1828596)
  * Add a reload command. (LP: #1828596)
  * kdump-config: implement try-reload (LP: #1828596)
  * udev: hotplug: use try-reload (LP: #1828596)
  * Use reset_devices as a cmdline parameter. (LP: #1800566)

 -- Thadeu Lima de Souza Cascardo <email address hidden> Wed, 18 Dec 2019 16:06:16 -0300

Changed in makedumpfile (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.