kdump-tools is unable to resolve DNS when systemd-resolved is used

[Guilherme G. Piccoli]

[Impact]

* Currently kdump is unable to handle domain name resolution due to the lack of "resolved" service in the kdump environment; this happens given the nature of reduced service load required in the kdump scenario.

* Kdump currently relies on a systemd service approach - there are 2 services, one being a configuration/load entity (kdump-tools.service) whereas the other is the crash dump service itself (kdump-tools-dump.service). Due to the complexity and risk in booting a machine after a kernel crash to collect kernel dump, it's expected kdump-tools-dump to load the minimal possible amount of services. In order to achieve that, kdump-tools-dump relies only in the sysinit and network-online targets.

* The systemd DNS tool ("resolved") is not ready in the moment kdump-tools-dump service is up to collect the kernel dump; also, "resolved" depends on dbus.socket to work, so to have a fully functional DNS resolution we are hereby adding both services as dependencies for kdump-tools-dump, so the network dump functionality works with hostnames (not requiring anymore that users set IP raw addresses).

* The attached SVG files (kdump.svg and regular_boot.svg) contains "systemd-analyze plot" outputs from a kdump-tools-dump and regular boot perspective. To collect the kdump data we manage to change the kdump-tools-dump service to load SSHd and also that required disabling the OneShot property of such service. With that data, one can check the services started/completed in each environment - it's possible to notice the absence of systemd-resolved when in kdump environment.

* Notice this modification is being worked concurrently with other kdump-tools' changes in LP #1828596.

[Test Case]

1) Deploy a Bionic VM e.g. with uvt-kvm;
2) Set-up console output in the guest;
3) Install the kdump-tools package;
4) Configure a network dump (SSH or NFS) using hostnames for the target machine;
5) Perform the test dump (with 'echo c> /proc/sysrq-trigger') and watch the failure in name resolution,

[Regression Potential]

* The modifications hereby proposed are minimal and scope-constrained to kdump-tools package; it'll only affect the services loaded before kdump-tools-dump collecting service.
A regression would then potentially fails kdump completion if one of the new services added to the kdump environment load (resolved and dbus.socket) fails to load and stalls/prevents the kdump service complete load.

[Guilherme G. Piccoli (gpiccoli)]

Disco soon to be retired (in favor of Eoan) - given that the impact of this bug is medium to low, we won't apply the future changes to Disco.

[Guilherme G. Piccoli (gpiccoli)]

Contradicting my comment #1, we decided to also fix this in Disco.
Cheers,

Guilherme

[Launchpad Janitor (janitor)]

This bug was fixed in the package makedumpfile - 1:1.6.6-4ubuntu1

---------------
makedumpfile (1:1.6.6-4ubuntu1) focal; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Merge from Debian unstable. Remaining changes:
    - Bump amd64 crashkernel from 384M-:128M to 512M-:192M.
  * Use reset_devices as a cmdline parameter. (LP: #1800566)
  * Use kdump-config reload after cpu or memory hotplug. (LP: #1828596)

  [ Guilherme G. Piccoli ]
  * Add a systemd-resolved service dependency in order kdump-tools is able
    to resolve DNS when in kdump boot. (LP: #1856323)

makedumpfile (1:1.6.6-4) unstable; urgency=medium

  * Let the kernel decide the crashkernel offset for ppc64el (LP: #1741860)
  * kdump-config: implement try-reload
  * udev: hotplug: use try-reload
  * Set Rules-Requires-Root to no

makedumpfile (1:1.6.6-3) unstable; urgency=medium

  * Add a reload command.
  * Use kdump-config reload after cpu or memory hotplug.
  * Use reset_devices as a cmdline parameter.

 -- Thadeu Lima de Souza Cascardo <email address hidden> Wed, 18 Dec 2019 14:38:51 -0300

[Guilherme G. Piccoli (gpiccoli)]

For this LP SRU submission, the following candidate packages were tested in amd64 arch in a qemu guest for each release:
* Bionic, candidate version 1.6.5-1ubuntu1~18.04.4;
* Disco, candidate version 1.6.5-1ubuntu1.4;
* Eoan, candidate version 1.6.6-2ubuntu2;

The test consisted in trying a SSH dump to a hostname, with the kdump-tools present in -updates pocket, and see it fails (not being able to resolve the hostname). Then, install the candidate version in each release and observe the kdump complete successfully, being able to resolve hostnames.

Cheers,

Guilherme

[Andy Whitcroft (apw)]

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.6-2ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted makedumpfile (1:1.6.6-2ubuntu2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.6-2ubuntu2 (i386, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Andy Whitcroft (apw)]

Hello Guilherme, or anyone else affected,

Accepted makedumpfile into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/makedumpfile/1:1.6.5-1ubuntu1~18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted makedumpfile (1:1.6.5-1ubuntu1~18.04.4) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

makedumpfile/1:1.6.5-1ubuntu1~18.04.4 (ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#makedumpfile

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Guilherme G. Piccoli (gpiccoli)]

I've verified the Eoan version in -proposed (1:1.6.6-2ubuntu2). The test was to try a network/SSH dump with the previous eoan version (in -release pocket) and observe it fails, due to being unable to resolve the hostname set in the kdump-tools configuration. After that, I've upgraded to the eoan-proposed version and the SSH dump worked - it was able to resolve the hostname.

Cheers,

Guilherme

[Guilherme G. Piccoli (gpiccoli)]

I've verified the Bionic version in -proposed (1:1.6.5-1ubuntu1~18.04.4). The test was to try a network/SSH dump with the previous Bionic version (in -updates pocket, version 1:1.6.5-1ubuntu1~18.04.3) and observe it fails, due to being unable to resolve the hostname set in the kdump-tools configuration.
After that, I've upgraded to the bionic-proposed version and the SSH dump worked - it was able to resolve the hostname.

Cheers,

Guilherme

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for makedumpfile has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package makedumpfile - 1:1.6.6-2ubuntu2

---------------
makedumpfile (1:1.6.6-2ubuntu2) eoan; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Fixes for DLPAR cpu add operation (LP: #1828596)
    - d/kdump-config.in: Add a reload command.
    - d/kdump-config.in: implement try-reload.
    - d/50-kdump-tools.rules: Use kdump-config reload after cpu or memory hotplug
    - d/50-kdump-tools.rules: use try-reload instead.
  * d/rules: Use reset_devices as a cmdline parameter. (LP: #1800566)

  [ Guilherme G. Piccoli ]
  * d/kdump-tools-dump.service: Add a systemd-resolved service dependency
    in order to make kdump-tool able to resolve DNS when in kdump boot.
    (LP: #1856323)
  * d/p/0003-Increase-SECTION_MAP_LAST_BIT-to-4.patch: x86_64: Fix an error due
    to makedumpfile being out-of-sync with recent kernels. (LP: #1857616)

 -- <email address hidden> (Guilherme G. Piccoli) Fri, 03 Jan 2020 16:10:19 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package makedumpfile - 1:1.6.5-1ubuntu1~18.04.4

---------------
makedumpfile (1:1.6.5-1ubuntu1~18.04.4) bionic; urgency=medium

  [ Thadeu Lima de Souza Cascardo ]
  * Fixes for DLPAR cpu add operation (LP: #1828596)
    - d/kdump-config.in: Add a reload command.
    - d/kdump-config.in: implement try-reload.
    - d/50-kdump-tools.rules: Use kdump-config reload after cpu or memory hotplug
    - d/50-kdump-tools.rules: use try-reload instead.
  * d/rules: Use reset_devices as a cmdline parameter. (LP: #1800566)

  [ Guilherme G. Piccoli ]
  * d/kdump-tools-dump.service: Add a systemd-resolved service dependency
    in order to make kdump-tool able to resolve DNS when in kdump boot.
    (LP: #1856323)
  * Fix an error due to makedumpfile being out-of-sync with recent kernels.
    (LP: #1857616)
    - d/p/0004-x86_64-fix-get_kaslr_offset_x86_64-to-return-kaslr_offset-correctly.patch
    - d/p/0005-Increase-SECTION_MAP_LAST_BIT-to-4.patch

 -- <email address hidden> (Guilherme G. Piccoli) Fri, 03 Jan 2020 13:14:39 -0300