kdump-tools is unable to resolve DNS when systemd-resolved is used

Bug #1856323 reported by Guilherme G. Piccoli on 2019-12-13
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
makedumpfile (Ubuntu)
Status tracked in Focal
Bionic
Medium
Guilherme G. Piccoli
Disco
Undecided
Unassigned
Eoan
Medium
Guilherme G. Piccoli
Focal
Medium
Guilherme G. Piccoli

Bug Description

[Impact]

* Currently kdump is unable to handle domain name resolution due to the lack of "resolved" service in the kdump environment; this happens given the nature of reduced service load required in the kdump scenario.

* Kdump currently relies on a systemd service approach - there are 2 services, one being a configuration/load entity (kdump-tools.service) whereas the other is the crash dump service itself (kdump-tools-dump.service). Due to the complexity and risk in booting a machine after a kernel crash to collect kernel dump, it's expected kdump-tools-dump to load the minimal possible amount of services. In order to achieve that, kdump-tools-dump relies only in the sysinit and network-online targets.

* The systemd DNS tool ("resolved") is not ready in the moment kdump-tools-dump service is up to collect the kernel dump; also, "resolved" depends on dbus.socket to work, so to have a fully functional DNS resolution we are hereby adding both services as dependencies for kdump-tools-dump, so the network dump functionality works with hostnames (not requiring anymore that users set IP raw addresses).

* The attached SVG files (kdump.svg and regular_boot.svg) contains "systemd-analyze plot" outputs from a kdump-tools-dump and regular boot perspective. To collect the kdump data we manage to change the kdump-tools-dump service to load SSHd and also that required disabling the OneShot property of such service. With that data, one can check the services started/completed in each environment - it's possible to notice the absence of systemd-resolved when in kdump environment.

* Notice this modification is being worked concurrently with other kdump-tools' changes in LP #1828596.

[Test Case]

1) Deploy a Bionic VM e.g. with uvt-kvm;
2) Set-up console output in the guest;
3) Install the kdump-tools package;
4) Configure a network dump (SSH or NFS) using hostnames for the target machine;
5) Perform the test dump (with 'echo c> /proc/sysrq-trigger') and watch the failure in name resolution,

[Regression Potential]

* The modifications hereby proposed are minimal and scope-constrained to kdump-tools package; it'll only affect the services loaded before kdump-tools-dump collecting service.
A regression would then potentially fails kdump completion if one of the new services added to the kdump environment load (resolved and dbus.socket) fails to load and stalls/prevents the kdump service complete load.

Guilherme G. Piccoli (gpiccoli) wrote :

Disco soon to be retired (in favor of Eoan) - given that the impact of this bug is medium to low, we won't apply the future changes to Disco.

Changed in makedumpfile (Ubuntu Eoan):
status: New → Confirmed
Changed in makedumpfile (Ubuntu Bionic):
status: New → Confirmed
Changed in makedumpfile (Ubuntu Eoan):
importance: Undecided → Medium
Changed in makedumpfile (Ubuntu Disco):
importance: Undecided → Low
Changed in makedumpfile (Ubuntu Bionic):
importance: Undecided → Medium
Changed in makedumpfile (Ubuntu Eoan):
assignee: nobody → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Disco):
assignee: nobody → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Bionic):
assignee: nobody → Guilherme G. Piccoli (gpiccoli)
Changed in makedumpfile (Ubuntu Disco):
status: New → Won't Fix
Dan Streetman (ddstreet) on 2019-12-20
Changed in makedumpfile (Ubuntu Disco):
status: Won't Fix → In Progress
Changed in makedumpfile (Ubuntu Eoan):
status: Confirmed → In Progress
Changed in makedumpfile (Ubuntu Focal):
status: Confirmed → In Progress
Changed in makedumpfile (Ubuntu Bionic):
status: Confirmed → In Progress
Guilherme G. Piccoli (gpiccoli) wrote :

Contradicting my comment #1, we decided to also fix this in Disco.
Cheers,

Guilherme

Guilherme G. Piccoli (gpiccoli) wrote :
description: updated
Guilherme G. Piccoli (gpiccoli) wrote :
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.6-4ubuntu1

---------------
makedumpfile (1:1.6.6-4ubuntu1) focal; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Merge from Debian unstable. Remaining changes:
    - Bump amd64 crashkernel from 384M-:128M to 512M-:192M.
  * Use reset_devices as a cmdline parameter. (LP: #1800566)
  * Use kdump-config reload after cpu or memory hotplug. (LP: #1828596)

  [ Guilherme G. Piccoli ]
  * Add a systemd-resolved service dependency in order kdump-tools is able
    to resolve DNS when in kdump boot. (LP: #1856323)

makedumpfile (1:1.6.6-4) unstable; urgency=medium

  * Let the kernel decide the crashkernel offset for ppc64el (LP: #1741860)
  * kdump-config: implement try-reload
  * udev: hotplug: use try-reload
  * Set Rules-Requires-Root to no

makedumpfile (1:1.6.6-3) unstable; urgency=medium

  * Add a reload command.
  * Use kdump-config reload after cpu or memory hotplug.
  * Use reset_devices as a cmdline parameter.

 -- Thadeu Lima de Souza Cascardo <email address hidden> Wed, 18 Dec 2019 14:38:51 -0300

Changed in makedumpfile (Ubuntu Focal):
status: In Progress → Fix Released
Guilherme G. Piccoli (gpiccoli) wrote :

For this LP SRU submission, the following candidate packages were tested in amd64 arch in a qemu guest for each release:
* Bionic, candidate version 1.6.5-1ubuntu1~18.04.4;
* Disco, candidate version 1.6.5-1ubuntu1.4;
* Eoan, candidate version 1.6.6-2ubuntu2;

The test consisted in trying a SSH dump to a hostname, with the kdump-tools present in -updates pocket, and see it fails (not being able to resolve the hostname). Then, install the candidate version in each release and observe the kdump complete successfully, being able to resolve hostnames.

Cheers,

Guilherme

Changed in makedumpfile (Ubuntu Disco):
status: In Progress → Won't Fix
importance: Low → Undecided
assignee: Guilherme G. Piccoli (gpiccoli) → nobody

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.6-2ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in makedumpfile (Ubuntu Eoan):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-eoan

All autopkgtests for the newly accepted makedumpfile (1:1.6.6-2ubuntu2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.6-2ubuntu2 (i386, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.5-1ubuntu1~18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in makedumpfile (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic

All autopkgtests for the newly accepted makedumpfile (1:1.6.5-1ubuntu1~18.04.4) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.5-1ubuntu1~18.04.4 (ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

I've verified the Eoan version in -proposed (1:1.6.6-2ubuntu2). The test was to try a network/SSH dump with the previous eoan version (in -release pocket) and observe it fails, due to being unable to resolve the hostname set in the kdump-tools configuration. After that, I've upgraded to the eoan-proposed version and the SSH dump worked - it was able to resolve the hostname.

Cheers,

Guilherme

tags: added: verification-done-eoan
removed: verification-needed-eoan
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers