[Hyper-V] issues with linux-next on 15.04 kdump functionality

[Chris Valean]

Using linux-next 4.3.0-next-20151106 kdump is failing to start as a service.
On RHEL and SLES with upstream, kdump is working as expected so only Ubuntu has this issue as observed from our testing.

A similar issue with the same error message from kdump has been already reported here: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1400319
However that behavior was on 32bit Ubuntu only, and now with upstream this seems to extend to 64bit as well.
So I'm not sure if we should link these 2 bug reports at this point.

Issues description:
1. kdump-tools service fails to start on 15.04 64bit, when runnig the linux-next kernel
Error returned is: Starting kdump-tools: Could not find a free area of memory of 0xa637000 bytes...
(full log below)
By default, kdump sets crashkernel to 394M-:128M

2. If crashkernel is set to 384M, the dump file will fail to create due to OOM messages when a crash is triggered.
Attaching the full serial log with crashkernel=384M. The log is the same with 512M as crashkernel.
Can this be a memory leak in the kdump kernel, making even already high values as 512M for the crashkernel to not be enough?
VM RAM has been set to 2GB and also 4GB, and with minimal load on the system that isn't a problem.

Work-around for issue #1:
Set crashkernel size to 384M, instead of the default 384M-:128M.
With this, kdump-tools will start.
However the dump file doesn't get written and the system will hang, without rebooting.

Values used to trigger a kdump:
- 384M
- 512M
- 384M-:128M - kdump-tools will not properly load.

Full kdump service log:
root@ubuntu1504srv:~# service kdump-tools status
â kdump-tools.service - Kernel crash dump capture service
   Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2015-11-11 06:02:28 PST; 20min ago
  Process: 792 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS)
 Main PID: 792 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/kdump-tools.service

Nov 11 06:02:20 ubuntu1504srv systemd[1]: Starting Kernel crash dump capture service...
Nov 11 06:02:28 ubuntu1504srv kdump-tools[792]: Starting kdump-tools: Could not find a free area of memory of 0xa637000 bytes...
Nov 11 06:02:28 ubuntu1504srv kdump-tools[792]: locate_hole failed
Nov 11 06:02:28 ubuntu1504srv kdump-tools[792]: * failed to load kdump kernel
Nov 11 06:02:28 ubuntu1504srv systemd[1]: Started Kernel crash dump capture service.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1515301

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Chris Valean (cvalean)]

Relevant logs attached already.

[Joshua R. Poulson (jrp)]

Per bug 1519917 once the Hyper-V support is rebased to upstream's 4.3 version, 15.10 kdump will start failing in a similar (if not the same) way.

[Chris Valean (cvalean)]

Using a recent linux-next on top of 15.04, the default config generates a large memory footprint initrd - over 500MB.

Therefore, if we set the crashkernel to 1GB, we'll get a crashdump and the proper behavior.
So this doesn't relate to 1400319, and it seems to be resolved in linux-next in connection to kdump from Ubuntu.

[Joseph Salisbury (jsalisbury)]

Is this still an issue? If so, I'll research this bug in parallel to working on 1400319.

[Chris Valean (cvalean)]

This has been fixed in the meantime, the related issue with ubuntu & linux-next remains that a high crashkernel size must be used - 1GB@1GB even though the initrd image is now under 200MB.

Let's keep this separate from 1400319 as at this point the issues are different.
Thanks Joe!

[Chris Valean (cvalean)]

Initially we were using 15.04, which is now EOL.

Testing has been switched to 15.10, which running with upstream does no longer have the 1GB crashkernel requirement.
It works with the default 384M ( -:128M).

We can consider this bug as closed therefore.
Thanky you!