kdump service does not start after configure/reboot

Bug #1708409 reported by bugproxy
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Fix Released
High
Canonical Foundations Team
makedumpfile (Ubuntu)
Fix Released
Wishlist
Canonical Foundations Team
Artful
Fix Released
Wishlist
Unassigned
Bionic
Fix Released
Wishlist
Canonical Foundations Team
systemd (Ubuntu)
Fix Released
High
Canonical Foundations Team
Artful
Fix Released
High
Dimitri John Ledkov
Bionic
Fix Released
High
Canonical Foundations Team

Bug Description

== Comment: #0 - Harish Sriram <email address hidden> - 2017-08-02 01:45:01 ==
kdump service does not start after configure/reboot

--Problem Description---
kdump service does not start after configure/reboot. It has to be started/loaded manually, everytime after reboot.

# kdump-config status
current state : Not ready to kdump

---uname output---
Linux ltc-test-ci2 4.11.0-10-generic #15-Ubuntu SMP Thu Jun 29 15:02:54 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type/Model = Power 8/8247-22L

----Additional Info-----
# cat /proc/cmdline
root=UUID=974df602-c0e4-4e67-8853-78ad15884c59 ro console=tty0 console=ttyS0,115200 quiet splash cgroup_enable=memory swapaccount=1 crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M

---Steps to Reproduce---
1. installed linux-crashdump
2. edited the kdump-tools.cfg crashkernel cmdline to above
3. update-grub
4. reboot

Expected:
kdump-config to be loaded by default after reboot

# kdump-config status
current state : Not ready to kdump

# service kdump-tools status
* kdump-tools.service - Kernel crash dump capture service
   Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor pres
   Active: inactive (dead)

...........................
https://github.com/systemd/systemd/issues/6334

systemd in artful is not properly picking up the unit files in
/etc/systemd/system/default.target.wants

CVE References

Revision history for this message
bugproxy (bugproxy) wrote : sosreport

Default Comment by Bridge

tags: added: architecture-ppc64le bugnameltc-157224 severity-high targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → systemd (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Foundations Team (canonical-foundations)
importance: Undecided → High
Revision history for this message
Steve Langasek (vorlon) wrote :

The referenced upstream bug report shows that upstream agrees this is a regression in systemd behavior which they are planning to correct, which is reasonable.

However, I think this is also a bug in the kdump-tools package. 'default.target' is a meta-target, which no package can know whether it's appropriate to start its service under without knowing what default.target is set to. In particular, kdump-tools also declares 'Wants=network-online.target'. If default.target (either on the kernel commandline or on the filesystem) is pointed to some sort of local-only rescue target, having network-online pulled into this target via kdump-tools could break the boot.

I think kdump-tools in Debian and Ubuntu should instead be integrating with multi-user.target.

Changed in systemd (Ubuntu):
importance: Undecided → High
status: New → Triaged
Changed in makedumpfile (Ubuntu):
assignee: nobody → Canonical Foundations Team (canonical-foundations)
Changed in systemd (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Foundations Team (canonical-foundations)
Changed in ubuntu-power-systems:
assignee: Canonical Foundations Team (canonical-foundations) → nobody
Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
status: New → Triaged
Manoj Iyer (manjo)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Foundations Team (canonical-foundations)
Manoj Iyer (manjo)
tags: added: triage-a
bugproxy (bugproxy)
tags: added: severity-medium
removed: severity-high
tags: added: id-5983be8a574a2f44b3cbc1ed
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-10-03 03:07 EDT-------
Due to this issue, "fadump" on pVM does not take a dump because fadump reboot the machine first and takes a dump.

Once service is manually started after reboot, the fadump is being captured and system reboots again.

Thanks,
Harish

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in makedumpfile (Ubuntu):
status: New → Confirmed
tags: added: ppc64el-kdump
Revision history for this message
Thadeu Lima de Souza Cascardo (cascardo) wrote :

It seems this has been reverted on systemd 235. However, as this might come to bite us back again in the future (it was mentioned that dependencies should not be added to aliases), and as Steve has pointed out, it makes more sense to use multi-user.target, I am going to fix it in a future release to come soon.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Systemd regression introduced in:
$ git describe 2d058a87ffb2d31a50422a8aebd119bbb4427244
v232-626-g2d058a8

Reverted in:
$ git describe 9e4ea9cc34fa032a47c253ddd94ac6c7afda663e
v234-72-g9e4ea9c

Thus affected releases were artful and bionic.

Bionic is now on v235 and thus is fixed-release for the systemd portion of the bug.

I fear there might be other packages that are also affected by this, thus I'm going to cherrypick the revert into artful in systemd as well.

Changed in systemd (Ubuntu Bionic):
status: Triaged → Fix Released
Changed in systemd (Ubuntu Artful):
importance: Undecided → High
status: New → Confirmed
Changed in makedumpfile (Ubuntu Bionic):
importance: Undecided → Wishlist
Changed in makedumpfile (Ubuntu Artful):
importance: Undecided → Wishlist
Changed in systemd (Ubuntu Artful):
status: Confirmed → Fix Committed
status: Fix Committed → In Progress
assignee: nobody → Dimitri John Ledkov (xnox)
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-01-09 09:15 EDT-------
After boot on Bionic Beaver , issue is not observed.

# uname -a
Linux ltc-wspoon12 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr:
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinux-4.13.0-17-generic
kdump initrd:
/var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.13.0-17-generic
current state: ready to kdump

kexec command:
/sbin/kexec -p --command-line="root=UUID=f009e6ec-b67d-4e08-89c9-b35b2b9e657f ro quiet splash xmon=on irqpoll noirqdistrib nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

Dmesg
-----------
[ 32.866017] kdump-tools[3949]: Starting kdump-tools: Modified cmdline:root=UUID=f009e6ec-b67d-4e08-89c9-b35b2b9e657f ro quiet splash irqpoll noirqdistrib nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0 elfcorehdr=157184K
[ 33.277170] kdump-tools[3949]: * loaded kdump kernel

Regards,
Harish

bugproxy (bugproxy)
tags: added: targetmilestone-inin1804
removed: targetmilestone-inin---
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in makedumpfile (Ubuntu Artful):
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.3-1

---------------
makedumpfile (1:1.6.3-1) unstable; urgency=medium

  * New upstream release. Main new feature:
    o Support new kernels
     - The supported kernel is updated to 4.14.8 in this version. (LP: #1746299)
       4.14.9 and later can't be filtered correctly due to inappropriate VMCOREINFO,
       it will be resolved in kernel 4.15.

  * Changelog:
    o New feature
        - [PATCH v2] x86_64: Take care of init_level4_pgt rename in kernel (by Pratyush Anand) 64bd5db
        - [PATCH v2] Fix SECTION_MAP_MASK for kernel >= v.13 (by Pratyush Anand) 4bf4f2b
        - [PATCH v2] book3s/ppc64: Lower the max real address to 53 bits for kernels >= v4.11
          (by Bhupesh Sharma) 6c83e74
        - [PATCH v3 1/4] Support symbol __cpu_online_mask (by Takao Indoh) 3b11c00
        - [PATCH] ppc64: update hash page table geometry (by Hari Bathini) 3c39f13
        - [PATCH] handle renamed init_level4_pgt -> init_top_pgt (by Jeff Mahoney) 5e54d53

    o Bugfix
        - [PATCH v2] arm64: Fix page table walk of 1GB section (by Bradley Bolen) 27508f1
        - [PATCH v2 1/2] ppc64: set page_offset in get_versiondep_info_ppc64() (by Pingfan Liu) 52319d2
        - [PATCH v2 2/2] ppc64: get the info of mem reserved for crashkernel (by Pingfan Liu) c7fcbbc
        - [PATCH v3 3/4] sadump: Fix a KASLR problem of sadump (by Takao Indoh) b4f7d95
        - [PATCH v3 4/4] sadump: Fix a KASLR problem of sadump while kdump is working (by Takao Indoh) 13d3059
        - [PATCH 2/2] Fix 'kernel_version' variable being uninitialized & introduce minor reorganization
          (by Bhupesh Sharma) d1ffe82
        - [PATCH 1/2] Fix off-by-one errors in exclude_segment() (by Petr Tesarik) 590f35e
        - [PATCH 2/2] Fix physical-to-virtual conversion in exclude_segment() (by Petr Tesarik) 6c1bf2a

    o Cleanup
        - [PATCH] Fix formatting problems in header file (by Eric DeVolder) cefea9e
        - [PATCH v3 2/4] Introduce vtop4_x86_64_pagetable (by Takao Indoh) 6de5d37
        - [PATCH 1/2] Fix compilation warnings on ppc64/ppc64le platforms (by Bhupesh Sharma) 0df157a
        - [PATCH] Make good use of is_cyclic_region() (by Atsushi Kumagai) 6bfd7a3
        - [PATCH] Fix the regression about getting kernel version (by Atsushi Kumagai) 254c116
        - [PATCH] Update maintainers (by Atsushi Kumagai) 9d3147e

  * Use multi-user.target. (LP: #1708409)
  * kdump-tools Recommends initramfs-tools-core.
  * Add myself as uploader.
  * Move Vcs control fields to salsa.

 -- Thadeu Lima de Souza Cascardo <email address hidden> Tue, 06 Feb 2018 12:53:14 -0200

Changed in makedumpfile (Ubuntu Bionic):
status: Confirmed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted systemd into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/234-2ubuntu12.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Manoj Iyer (manjo)
Changed in ubuntu-power-systems:
status: Triaged → Incomplete
Revision history for this message
Thadeu Lima de Souza Cascardo (cascardo) wrote :

The systemd update fixes the problem for me on artful.

tags: added: verification-done-artful
removed: verification-needed-artful
tags: added: verification-done
removed: verification-needed
Manoj Iyer (manjo)
Changed in ubuntu-power-systems:
status: Incomplete → Fix Committed
tags: added: triage-g
removed: triage-a
Changed in makedumpfile (Ubuntu Artful):
status: Confirmed → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 234-2ubuntu12.3

---------------
systemd (234-2ubuntu12.3) artful; urgency=medium

  [ Dimitri John Ledkov ]
  * Fix test-functions failing with Ubuntu units. LP: #1750608
  * tests: switch to using ext4 by default, instead of ext3. LP: #1750608
  * Fix kdump service not starting, due to systemd not loading dropins.
    Cherrypick a fix from upstream. (LP: #1708409)
  * systemd-fsckd: Fix ADT tests to work on s390x too. (LP: #1736955)
  * netwokrd: add support for RequiredForOnline stanza. (LP: #1737570)
  * resolved.service: set DefaultDependencies=no (LP: #1734167)
  * systemd.postinst: enable persistent journal. (LP: #1618188)
  * core: add support for non-writable unified cgroup hierarchy for container support.
    Rebase and de-fuzz. (LP: #1734410)
  * Prevent MemoryDenyWriteExecution policy bypass, by disallowing pkey_mprotect when mprotect is disallowed.
    CVE-2017-15908 (LP: #1725348)
  * networkd: enable promote_secondaries on networkd managed dhcp links.
    This fixes failing to renew DHCP lease, on networkd managed devices.
    (LP: #1721223)

  [ Kleber Sacilotto de Souza ]
  * systemd-rfkill service times out when a new rfkill device is added
    - rfkill-fix-erroneous-behavior-when-polling-the-udev-.patch: Comparing
    udev_device_get_sysname(device) and sysname will always return true. We need to
    check the device received from udev monitor instead.
    - rfkill-fix-typo.patch: Fix typo in rfkill log message. (LP: #1734908)

 -- Dimitri John Ledkov <email address hidden> Tue, 20 Feb 2018 16:11:58 +0000

Changed in systemd (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Changed in ubuntu-power-systems:
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package makedumpfile - 1:1.6.3-2~17.10.1

---------------
makedumpfile (1:1.6.3-2~17.10.1) artful; urgency=medium

  * Backport latest makedumpfile to artful. (LP: #1746299)

 -- Thadeu Lima de Souza Cascardo <email address hidden> Mon, 21 May 2018 10:56:29 -0300

Changed in makedumpfile (Ubuntu Artful):
status: Invalid → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.