kdump functionality not working as expected when /boot is a separate partition

Bug #828731 reported by Louis Bouchard on 2011-08-18
32
This bug affects 6 people
Affects Status Importance Assigned to Milestone
kexec-tools (Ubuntu)
Medium
Canonical Foundations Team
Lucid
Medium
Barry Warsaw
Maverick
Medium
Unassigned
Natty
Medium
Barry Warsaw
Oneiric
Medium
Barry Warsaw

Bug Description

SRU Request for Lucid/Maverick/Natty/Oneiric:

[Impact] - When a server is configured with the /boot as a separate partition, which is the default configuration when LVM installation is selected, the kdump mechanism fails systematically.

[Development/Stable Fix] - This has been fixed in the development version with the addition of the following:

  * Backport changes to fix kdump functionality. LP: #828731.
    - debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
      so that if makedumpfile is statically linked, we get proper library
      resolution. Thanks to Louis Bouchard <email address hidden> for
      the patch. LP: #785425.
    - debian/kdump.initramfs: handle the possibility that /usr, /boot, or
      /var is on a separate filesystem and needs to be manually mounted before
      calling makedumpfile. LP: #828731.
    - Depend on makedumpfile, without which the initramfs script doesn't work.
    - Fix an unnecessary bashism.
    - Only install the kdump initramfs script and depend on makedumpfile on
      architectures that makedumpfile supports.

[Test Case] - See 'how to reproduce' below.
[Regression Potential] - Need to ensure kdump works correctly in backported versions.

--

Description : Ubuntu 10.04.2
Release : 10.04

When a server is configured with the /boot as a separate partition, which is the default configuration when LVM installation is selected, the kdump mechanism fails systematically.

This is caused by the fact that the ./scripts/init-bottom/0_kdump script that is loaded into the initrd.img file make the assumption that /boot is _ALWAYS_ a directory which contains the vmcoreinfo-$KVER file. The bug is contained within the following code :

   KVER="`uname -r`"
   INFO="$rootmnt/boot/vmcoreinfo-$KVER"
   CRASHFILE="$rootmnt/var/crash/vmcore"
   MAKEDUMPFILE="$rootmnt/usr/bin/makedumpfile"
   LOG="$rootmnt/var/crash/vmcore.log"
   VMCORE="/proc/vmcore"

   # Check that this is a kexec kernel.
   grep -q kdump_needed /proc/cmdline || exit 0

   # Do NOT exit the script after this point, or the system will start
   # booting inside the crash kernel.

   . ./scripts/functions

   # Make sure makedumpfile assumptions are satisfied.
   while ! test -e "$INFO"; do
           panic "kdump: Missing $INFO"
   done
   while ! test -x "$MAKEDUMPFILE"; do
           panic "kdump: Missing $MAKEDUMPFILE"
   done

The test 'while !test -e "$INFO";do' fails if /boot is a separate partition.

Reproducible: 100%

How to Reproduce :

Pre-requisite : a system or VM installed with LVM and /boot as a separate partition (default option for LVM installation)

1) install the linux-crashdump package & dependancies
2) Increase the crashkernel= parameter to 128M if the RAM is below 2048M (LP Bug#785394) in /etc/grub.d/10_linux
3) Run sudo update-grub
4) Reboot the system
5) Force a panic with "echo c > /proc/sysrq-trigger

The system will reboot to the kexec kernel with complete network access enabled :

 # cat /proc/cmdline
 BOOT_IMAGE=/vmlinuz-2.6.32-28-server root=/dev/mapper/Lucid--lvmS-root ro kdump_needed maxcpus=1 irqpoll reset_devices memmap=exactmap memmap=640K@0K memmap=130412K@33408K elfcorehdr=163820K

Workaround:
Copy the content of the /boot partition into the /boot directory. This is only valid until the next upgrade of the "linux-image-{version}" package.

How to workaround :

6) Reboot the system
7) Copy the content of the /boot partition into the /boot directory
   # df /boot
   Filesystem 1K-blocks Used Available Use% Mounted on
   /dev/vda1 233191 17563 203187 8% /boot
   # sudo umount /boot
   # sudo mount /dev/vda1 /mnt
   # sudo cp -pr /mnt/* /boot
   # sudo umount /mnt
   # sudo mount -a
   # sudo echo c > /proc/sysrq-tgrigger

The system will correctly generate a crash dump
   # find /var/crash
    /var/crash
    /var/crash/linux-image-2.6.32-28-server.0.crash

ProblemType: Bug
DistroRelease: Ubuntu 10.04.02
Package: kexec-tools-1-2.0.1-1ubuntu3
Uname: Linux 2.6.32-28-server x86_64
Architecture: amd64

Louis Bouchard (louis) wrote :

The following patch fixes the situation where the kdump data collection fails when the /boot is a separate partition, which is the case when the standard LVM installation is selected.

The patch has been built against the kexec-tools 1:2.0.2-1ubuntu3 (11.10 Oneiric) but also applies to the version found on Lucid. This was done to take into accound the next upcoming version with will be LTS.

Changed in kexec-tools (Ubuntu):
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Changed in kexec-tools (Ubuntu):
assignee: Canonical Kernel Team (canonical-kernel-team) → Canonical Foundations Team (canonical-foundations)

The attachment "kdump.initramfs_boot_partition-LP828731.patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Louis Bouchard (louis) wrote :

FYI

The patch has been tested on i386 and x86_64 in conjunction with the patch provided for LP#785425 which relies on this patch to work.

Both are confirmed to work and should be adressed together as they touch the same script.

James Downs (jdowns) wrote :

This also affects 10.10 Maverick.

James Downs (jdowns) wrote :

Also, the problem affects other filesystems that are separate partitions, specifically, /var

Louis Bouchard (louis) wrote :

@James

This affects all distro since Lucid which use a separate /boot partition, which is the default configuration for systems installed with LVM.

Now taking into account separate /var, /usr is a bit different, as this is not a standard way of installing Ubuntu, even though it is totally possible. Maybe adding the logic to take /usr and/or /var into account should be added in the script, but this should be something decided by the foundation team as it goes beyond bug fixing.

James Downs (jdowns) wrote :

@louis sure, but shouldn't these bugs be tagged against various releases so that it's possible to search for a bug and release, and find out whether it's fixed in each release?

Changed in kexec-tools (Ubuntu Lucid):
status: New → Triaged
Changed in kexec-tools (Ubuntu Maverick):
status: New → Triaged
Changed in kexec-tools (Ubuntu Natty):
status: New → Triaged
Changed in kexec-tools (Ubuntu Oneiric):
status: New → Triaged
Changed in kexec-tools (Ubuntu Lucid):
importance: Undecided → Medium
Changed in kexec-tools (Ubuntu Maverick):
importance: Undecided → Medium
Changed in kexec-tools (Ubuntu Natty):
importance: Undecided → Medium
Changed in kexec-tools (Ubuntu Oneiric):
importance: Undecided → Medium
Louis Bouchard (louis) wrote :

Here is the new version of the patch, without any reference to ext2

Chris J Arges (arges) wrote :

@louis:

Just wanted to make sure this new patch was verified and tested as well. Let me know if this has been done.

--chris

Louis Bouchard (louis) wrote :

@chris

Well, no it wasn't apparently :-( Testing with a newly created VM with /boot as XFS fails. I'll work on an update of the patch right away.

Louis Bouchard (louis) wrote :

@chris

Here is a new version of the patch, which takes into account any type of FS type for /boot. It extracts the type from the /etc/fstab which is then used with the mount command.

I have tested it with /boot as :
  - ext2 (standard)
  - xfs
  - jfs

All those three did generate a vmcore file.

Steve Langasek (vorlon) wrote :

Louis,

Attached is an untested patch for kexec-tools that should ensure that all of /boot, /var, and /usr are mounted correctly in the target system. Could you test that this patch works for you? If so, I'll push this to precise.

It should solve a number of problems related to trying to mount arbitrary filesystems (e.g., /boot) from the initramfs.

Louis Bouchard (louis) wrote :

Steve,

I'm preparing a test environment and will let you know how the patch behaves as soon as I can test it.

Louis Bouchard (louis) wrote :

Steve,

I am currently testing a slightly modified version of your patch which seems to work to some extent. Multiple partitions (i.e. /boot, /usr and /var) do work, depending on the type of file system used. A mix of ext2, ext4, xfs work fine but I currently have a failure when /var is jfs : for some reason, but initramfs environment refuses to mount /var as jfs, even if the jfs kernel module is present & loaded.

I will carry on with my test when I return for the xmas vacation in early jan. I'll keep you posted with my findings. Let me know if you want me to post the current version of the patch here.

Hi Louis,

On Thu, Dec 22, 2011 at 10:56:00AM -0000, Louis Bouchard wrote:

> Let me know if you want me to post the current version of the patch here.

As I wasn't expecting modifications to be needed, yes, please do :)

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Louis Bouchard (louis) wrote :

Hi Steve,

One addition is to check fstab for LVM usage. If used, 'mount --bind' the /dev to gain access to the /dev/mapper entries.

The second thing I found out is that makedumpfile requires /tmp to be writable, so it is not possible to restrict / to be read-only.

Here is the modified patch.

I have done some more tests. I still have the issue when /var is a JFS file system. When switching /var to ext4, the patch works well. For some reason, busybox has problems mounting JFS file systems. Right now, here is what I exhave tested :
/boot XFS
/ ext4
/usr XFS
/var JFS (nok), ext4 (ok)

Merry christmas & a very happy new year.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.2-3ubuntu2

---------------
kexec-tools (1:2.0.2-3ubuntu2) precise; urgency=low

  * debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
    so that if makedumpfile is statically linked, we get proper library
    resolution. Thanks to Louis Bouchard <email address hidden> for
    the patch. LP: #785425.
  * debian/kdump.initramfs: handle the possibility that /usr, /boot, or
    /var is on a separate filesystem and needs to be manually mounted before
    calling makedumpfile. LP: #828731.
  * Depend on makedumpfile, without which the initramfs script doesn't work.
 -- Steve Langasek <email address hidden> Mon, 02 Jan 2012 22:43:33 -0800

Changed in kexec-tools (Ubuntu):
status: Triaged → Fix Released
Chris J Arges (arges) wrote :

@steve, @louis:

I assume this fix needs to be applied to lucid/natty/oneiric as well? What would be the next step in that SRU process?
Thanks,
--chris

On Jan 9, 2012, at 8:15 AM, Chris J Arges wrote:

> I assume this fix needs to be applied to lucid/natty/oneiric as well? What would be the next step in that SRU process?

Maverick, too.

Chris J Arges (arges) wrote :

This patch backports the fix to the latest lucid release. This applies cleanly and I built an amd64 version, but did not test.

description: updated
Chris J Arges (arges) wrote :

This patch applies cleanly to maverick.

Chris J Arges (arges) wrote :

This patch applies cleanly to natty.

Chris J Arges (arges) wrote :

This patch applies cleanly to oneiric.

Changed in kexec-tools (Ubuntu Lucid):
assignee: nobody → Chris J Arges (christopherarges)
Changed in kexec-tools (Ubuntu Maverick):
assignee: nobody → Chris J Arges (christopherarges)
Changed in kexec-tools (Ubuntu Natty):
assignee: nobody → Chris J Arges (christopherarges)
Changed in kexec-tools (Ubuntu Oneiric):
assignee: nobody → Chris J Arges (christopherarges)
Louis Bouchard (louis) wrote :

@everybody

FYI, I have tested kexec-tools from Precise on the following :

 - Lucid
 - Maverick
 - Natty
 - Oneiric

kdump works as expected on all of those with the kexec-tool package from precise.

James Downs (jdowns) wrote :

I loaded this version of kexec-tools, and triggered a crash, but /boot and /var is not mounted..

Louis Bouchard (louis) wrote :

@james

Most probably, the initramfs did not get rebuilt after updating the package. Most of the time when I test, I issue :

initramfs -u
update-grub

then I reboot. You might want to rebuild the initramfs just in case and test again.

James Downs (jdowns) wrote :

Ahh, that's probably it. I thought I saw the installer rebuild it. I'll report back when I have a chance to test it on some other machines.

Is anyone doing any work to support streaming dumpfiles to a remote location?

Chris J Arges (arges) on 2012-01-31
Changed in kexec-tools (Ubuntu Lucid):
status: Triaged → In Progress
Changed in kexec-tools (Ubuntu Natty):
status: Triaged → In Progress
Changed in kexec-tools (Ubuntu Maverick):
status: Triaged → In Progress
Changed in kexec-tools (Ubuntu Oneiric):
status: Triaged → In Progress

The debdiffs generally look good to me, but I don't have sufficient permission to upload this package. Since you're also closing bug 785425 in the debdiffs, I'd also make sure the SRU justification and the ubuntu-sru, ubuntu-sponsors teams are subscribed and the releases properly nominated/targetted.

Chris J Arges (arges) wrote :

Ok I believe I've targeted things correctly. Any other actions to progress with the SRU?

Changed in kexec-tools (Ubuntu Lucid):
milestone: none → lucid-updates
Changed in kexec-tools (Ubuntu Maverick):
milestone: none → maverick-updates
Changed in kexec-tools (Ubuntu Natty):
milestone: none → natty-updates
Changed in kexec-tools (Ubuntu Oneiric):
milestone: none → oneiric-updates
Steve Langasek (vorlon) on 2012-02-23
Changed in kexec-tools (Ubuntu Lucid):
assignee: Chris J Arges (christopherarges) → Barry Warsaw (barry)
Changed in kexec-tools (Ubuntu Maverick):
assignee: Chris J Arges (christopherarges) → Barry Warsaw (barry)
Changed in kexec-tools (Ubuntu Natty):
assignee: Chris J Arges (christopherarges) → Barry Warsaw (barry)
Changed in kexec-tools (Ubuntu Oneiric):
assignee: Chris J Arges (christopherarges) → Barry Warsaw (barry)
Barry Warsaw (barry) wrote :

I'm going to sponsor these patches into *-proposed.

Barry Warsaw (barry) on 2012-02-24
tags: added: verification-needed
Changed in kexec-tools (Ubuntu Oneiric):
status: In Progress → Fix Committed
Changed in kexec-tools (Ubuntu Natty):
status: In Progress → Fix Committed
Changed in kexec-tools (Ubuntu Maverick):
status: In Progress → Fix Committed
Changed in kexec-tools (Ubuntu Lucid):
status: In Progress → Fix Committed
Steve Langasek (vorlon) wrote :

"Fix Committed"/"verification-needed" are auto-added once the package is accepted into -proposed.

Changed in kexec-tools (Ubuntu Lucid):
status: Fix Committed → In Progress
Changed in kexec-tools (Ubuntu Maverick):
status: Fix Committed → In Progress
Changed in kexec-tools (Ubuntu Natty):
status: Fix Committed → In Progress
Changed in kexec-tools (Ubuntu Oneiric):
status: Fix Committed → In Progress
tags: removed: verification-needed

Hello Louis, or anyone else affected,

Accepted kexec-tools into oneiric-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in kexec-tools (Ubuntu Oneiric):
status: In Progress → Fix Committed
tags: added: verification-needed
Evan Broder (broder) wrote :

Unsubscribing ubuntu-sponsors as all of the SRUs have been sponsored, and are just awaiting ubuntu-sru review.

Martin Pitt (pitti) wrote :

Hello Louis, or anyone else affected,

Accepted kexec-tools into lucid-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in kexec-tools (Ubuntu Lucid):
status: In Progress → Fix Committed
Martin Pitt (pitti) on 2012-02-28
Changed in kexec-tools (Ubuntu Maverick):
status: In Progress → Won't Fix
milestone: maverick-updates → none
assignee: Barry Warsaw (barry) → nobody
Louis Bouchard (louis) wrote :

Hello Martin, Barry,

I have tested the kexec-tools in oneiric-proposed and can confirm that the new package does fix the issue (tested on 11.10 x86_64 only).

Now waiting for the Lucid & Natty packages to test as well.

Martin Pitt (pitti) wrote :

Louis, it's already in lucid-proposed.

tags: added: verification-done
Steve Langasek (vorlon) wrote :

lucid/natty were apparently supposed to be there but weren't. Accepted now.

Changed in kexec-tools (Ubuntu Natty):
status: In Progress → Fix Committed
Louis Bouchard (louis) wrote :

@steve @martin

I have tested all releases and found an issue with the Lucid in -proposed.

The lucid-proposed-lp828731.debdiff is different from the others and has a couple of extra lines that break the fix which works for other releases. So the version in lucid-proposed should be removed as it _DOES NOT_ fix the problem.

For the other releases (Maverick, Natty, Oneiric and Precise), the patch has been tested and works correctly.

Steve Langasek (vorlon) wrote :

> The lucid-proposed-lp828731.debdiff is different from the others and has
> a couple of extra lines that break the fix which works for other
> releases. So the version in lucid-proposed should be removed as it
> _DOES NOT_ fix the problem.

Right, I've removed this now from lucid-proposed to ensure it doesn't get
promoted by accident.

Changed in kexec-tools (Ubuntu Lucid):
status: Fix Committed → In Progress

On Mar 01, 2012, at 06:43 PM, Steve Langasek wrote:

>> The lucid-proposed-lp828731.debdiff is different from the others and has
>> a couple of extra lines that break the fix which works for other
>> releases. So the version in lucid-proposed should be removed as it
>> _DOES NOT_ fix the problem.
>
>Right, I've removed this now from lucid-proposed to ensure it doesn't get
>promoted by accident.

I'll take another look at the Lucid SRU.

Barry Warsaw (barry) wrote :

Please specify the lines that are incorrect in the Lucid debdiff. Is it these two from kdump.initramfs?

@@ -24,22 +24,50 @@
 grep -q kdump_needed /proc/cmdline || exit 0

 # Make sure makedumpfile assumptions are satisfied.
-test -e $INFO || exit 0
-test -x $MAKEDUMPFILE || exit 0
+test -e $rootmnt/$INFO || exit 0
+test -x $rootmnt/$MAKEDUMPFILE || exit 0

Should the lines be removed or substituted with something else? Better yet, why not attach a diff against the current lucid-proposed version? :) I'd be happy to sponsor it again.

Louis Bouchard (louis) wrote :

@barry

From what I've seen, they're the only difference between Lucid and the other debdiff. For sure, it is indeed what is causing the failure in Lucid (those lines are not in the other versions). I'll have a closer look at it tomorrow morning and will submit a new debdiff for Lucid. I just want to be sure that we get all ends covered this time. From what I could gather, they should be removed but, moreover, the same patch should apply to all versions.

Steve Langasek (vorlon) wrote :

On Thu, Mar 01, 2012 at 08:14:44PM -0000, Louis Bouchard wrote:
> >From what I've seen, they're the only difference between Lucid and the
> other debdiff. For sure, it is indeed what is causing the failure in
> Lucid (those lines are not in the other versions).

The lines are there in the later patches, but take a different form in later
releases:

 # Make sure makedumpfile assumptions are satisfied.
-while ! test -e "$INFO"; do
+while ! test -e "$rootmnt/$INFO"; do
        panic "kdump: Missing $INFO"
 done
-while ! test -x "$MAKEDUMPFILE"; do
- panic "kdump: Missing $MAKEDUMPFILE"
+
+while ! test -x "$rootmnt/$MAKEDUMPFILE"; do
+ panic "kdump: Missing $rootmnt/$MAKEDUMPFILE"
 done

So it's not clear to me why those lines would cause failures for you on
lucid.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Louis Bouchard (louis) wrote :

@steve

The reason why it fails on Lucid is that those line are placed BEFORE the $rootmnt gets mounted as opposed to AFTER the mount for the other distro.

It is like that because Lucid runs an older version of the package (1:2.0.1-1) compared to 1:2.0.1-2 for maverick and natty and 1:2.0.2-1 for Oneiric.

What I have done is to have the kdump.initramfs for Lucid be identical to the ones of the other versions (merely move those lines near the end), recreated the debdiff and rebuilt a test package with it. It now works correctly on Lucid too. But I'm questioning the validity of those steps : shouldn't we just provide 1:2.0.1-2 on Lucid just like for maverick and Natty ?

If not, then I have the debdiff ready for upload so a respin of the lucid-proposed can be done.

Just let me know

Louis Bouchard (louis) wrote :

FYI,

After further investigation, the difference comes from LP: #600051 as noted in Maverick's kexec-tool changelog :

  * kdump.initramfs: panic if assumptions are not satisfied, instead of
    letting the system continue to boot inside the crash kernel. (LP: #600051)

Steve Langasek (vorlon) wrote :

On Fri, Mar 02, 2012 at 10:36:10AM -0000, Louis Bouchard wrote:
> The reason why it fails on Lucid is that those line are placed BEFORE
> the $rootmnt gets mounted as opposed to AFTER the mount for the other
> distro.

Aha, right - so these lines should be moved, rather than dropped.

> What I have done is to have the kdump.initramfs for Lucid be identical
> to the ones of the other versions (merely move those lines near the
> end), recreated the debdiff and rebuilt a test package with it. It now
> works correctly on Lucid too. But I'm questioning the validity of those
> steps : shouldn't we just provide 1:2.0.1-2 on Lucid just like for
> maverick and Natty ?

That depends on what else has changed in the meantime and whether all the
changes are suitable to include in an SRU.

> If not, then I have the debdiff ready for upload so a respin of the
> lucid-proposed can be done.

Easiest would be to make the one change to move the check below the mount.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Louis Bouchard (louis) wrote :

@steve

Moving the lines further down is what we get when running the same patch on the kdump.initramfs scripts of newer versions. So I have created a new debdiff for Lucid which produces the same kdump.initramfs for all versions.

The end result is that all distro will share the same kdump.initramfs script. I have positive tests with a new package built from that debdiff. Will be waiting for the -proposed version.

P.S. I have replaced the former lucid-proposed-lp828731.debdiff by a newer one.

Louis Bouchard (louis) wrote :

lucid-proposed-lp828731.debdiff

Hello Louis, or anyone else affected,

Accepted kexec-tools into lucid-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in kexec-tools (Ubuntu Lucid):
status: In Progress → Fix Committed
tags: removed: verification-done
Louis Bouchard (louis) wrote :

@clint

I have tested the new kexec-tool in lucid-proposed and it does work properly.

kexec-tools is now performing as expected on all supported distros (from Lucid onward). Let's hope that it makes it to -updates soon.

Thanks very much for your invaluable help.

Steve Langasek (vorlon) on 2012-03-15
tags: added: verification-done
tags: removed: verification-done
tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.1-1ubuntu5

---------------
kexec-tools (1:2.0.1-1ubuntu5) lucid-proposed; urgency=low

  * Backport changes to fix kdump functionality. LP: #828731.
    - debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
      so that if makedumpfile is statically linked, we get proper library
      resolution. Thanks to Louis Bouchard <email address hidden> for
      the patch. LP: #785425.
    - debian/kdump.initramfs: handle the possibility that /usr, /boot, or
      /var is on a separate filesystem and needs to be manually mounted before
      calling makedumpfile. LP: #828731.
    - Depend on makedumpfile, without which the initramfs script doesn't work.
    - Fix an unnecessary bashism.
    - Only install the kdump initramfs script and depend on makedumpfile on
      architectures that makedumpfile supports.
 -- Louis Bouchard <email address hidden> Mon, 05 Mar 2012 15:10:26 -0000

Changed in kexec-tools (Ubuntu Lucid):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.1-2ubuntu2.2

---------------
kexec-tools (1:2.0.1-2ubuntu2.2) maverick-proposed; urgency=low

  * Backport changes to fix kdump functionality. LP: #828731.
    - debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
      so that if makedumpfile is statically linked, we get proper library
      resolution. Thanks to Louis Bouchard <email address hidden> for
      the patch. LP: #785425.
    - debian/kdump.initramfs: handle the possibility that /usr, /boot, or
      /var is on a separate filesystem and needs to be manually mounted before
      calling makedumpfile. LP: #828731.
    - Depend on makedumpfile, without which the initramfs script doesn't work.
    - Fix an unnecessary bashism.
    - Only install the kdump initramfs script and depend on makedumpfile on
      architectures that makedumpfile supports.
 -- Chris J Arges <email address hidden> Wed, 18 Jan 2012 15:32:08 -0600

Changed in kexec-tools (Ubuntu Maverick):
status: Won't Fix → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.1-2ubuntu5

---------------
kexec-tools (1:2.0.1-2ubuntu5) natty-proposed; urgency=low

  * Backport changes to fix kdump functionality. LP: #828731.
    - debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
      so that if makedumpfile is statically linked, we get proper library
      resolution. Thanks to Louis Bouchard <email address hidden> for
      the patch. LP: #785425.
    - debian/kdump.initramfs: handle the possibility that /usr, /boot, or
      /var is on a separate filesystem and needs to be manually mounted before
      calling makedumpfile. LP: #828731.
    - Depend on makedumpfile, without which the initramfs script doesn't work.
    - Fix an unnecessary bashism.
    - Only install the kdump initramfs script and depend on makedumpfile on
      architectures that makedumpfile supports.
 -- Chris J Arges <email address hidden> Wed, 18 Jan 2012 17:05:10 -0600

Changed in kexec-tools (Ubuntu Natty):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kexec-tools - 1:2.0.2-1ubuntu4

---------------
kexec-tools (1:2.0.2-1ubuntu4) oneiric-proposed; urgency=low

  * Backport changes to fix kdump functionality. LP: #828731.
    - debian/kdump.initramfs: call /usr/bin/makedumpfile via a chroot command,
      so that if makedumpfile is statically linked, we get proper library
      resolution. Thanks to Louis Bouchard <email address hidden> for
      the patch. LP: #785425.
    - debian/kdump.initramfs: handle the possibility that /usr, /boot, or
      /var is on a separate filesystem and needs to be manually mounted before
      calling makedumpfile. LP: #828731.
    - Depend on makedumpfile, without which the initramfs script doesn't work.
    - Fix an unnecessary bashism.
    - Only install the kdump initramfs script and depend on makedumpfile on
      architectures that makedumpfile supports.
 -- Chris J Arges <email address hidden> Wed, 18 Jan 2012 14:52:58 -0600

Changed in kexec-tools (Ubuntu Oneiric):
status: Fix Committed → Fix Released
Terry Wang (terrywang) wrote :

Is this really fixed on Ububuntu 14.04 LTS?

I've just tested triggering a crash dump and it didn't work. A Google search pointed me to this bug. When I see /boot on a separate partition, I just realized it is the damn case! Others like dbgsym kernel from ddeb, kdump-tools, linux-crashdump all installed and properly configured.

The VMs use LVM so they all have a separate /boot partition, that's why kdump was not able to dump the /proc/vmcore.

After using the workaround (umount /boot and use the /boot on /), it worked!

However, there is no 0_dump file under /usr/share/initramfs-tools/scripts/init-bottom/ WTF? So what is causing the problem in Ubuntu 14.04 with a separate /boot partition?

A separate bug needed?

Nagaraj S D (nagarajsdjkd) wrote :

I have a setup with Ubuntu 14.04 LTS server. The /boot is not on a separate partition. I followed the kdump setup as mentioned in https://help.ubuntu.com/14.04/serverguide/kernel-crash-dump.html

After triggering the kdump, system panics but system never comeup/reboot. It get stuck at the call trace of the panic.

Some of the commands output is below it might help to understand the issue:
root@ubuntu:~# mount
/dev/mapper/350002ac01d6314e1-part1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
none on /sys/fs/pstore type pstore (rw)
systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)
root@ubuntu:~#

hbauser@ubuntu:~$ ls /boot/
abi-3.13.0-24-generic grub memtest86+.bin memtest86+_multiboot.bin vmlinuz-3.13.0-24-generic
config-3.13.0-24-generic initrd.img-3.13.0-24-generic memtest86+.elf System.map-3.13.0-24-generic
hbauser@ubuntu:~$

hbauser@ubuntu:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.13.0-24-generic root=UUID=0503c0b3-1acd-4e11-a60a-f5f2618fdb6c ro crashkernel=384M-:128M
hbauser@ubuntu:~$ dmesg | grep -i crash
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.13.0-24-generic root=UUID=0503c0b3-1acd-4e11-a60a-f5f2618fdb6c ro crashkernel=384M-:128M
[ 0.000000] Reserving 128MB of memory at 720MB for crashkernel (System RAM: 32733MB)
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.13.0-24-generic root=UUID=0503c0b3-1acd-4e11-a60a-f5f2618fdb6c ro crashkernel=384M-:128M
hbauser@ubuntu:~$

In another configuration where I have installed OS with LVM and /boot on separate partition. The workaround suggested in this bug also didn't work. Am I missing something?

Anders Kaseorg (andersk) wrote :

Nagaraj, I see no reason to think your problem is related to this bug. This bug is specifically about a separate /boot partition, which you do not have, and your symptoms don’t match either. You should file a new report.

Louis Bouchard (louis) wrote :

First of all, doing ponytail on a 3 year old bug is not going to bring you much attention.

Especially since this bug pertains to a mechanism that is now only used on Precise. It is unfortunate though that the server guide is misleading on how to configure kernel dump. I would suggest to open a bug against the server guide to have this fixed.

This page does a pretty good job at documenting how to generate a kernel crash dump :

 https://wiki.ubuntu.com/Kernel/CrashdumpRecipe

TL;DR : Once the linux-crashdump package has been installed & the server rebooted you need to change the following in /etc/default/kdump-tools :

USE_KDUMP=1

Then :

 $ sudo kdump-config load

For any other visiting this bug, please refrain from adding to it, it is no longer relevant as the kernel dump mechanism has changed

vigilian (vigilian) wrote :

same bug than before in trusty.
/boot on sde5
and / is on sdd2

can you release a fix please. I need to evaluate my kernel panic FAST! thank you

Claudio Corvino (claud81) wrote :

still present in trusty! /boot in sda1 and / is on LVM

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers