makedumpfile falls back to cp with "__vtop4_x86_64: Can't get a valid pmd_pte."
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
makedumpfile (Ubuntu) |
Fix Released
|
Medium
|
Kellen Renshaw | ||
Focal |
Fix Released
|
Medium
|
Heather Lemon |
Bug Description
[Impact]
* On Focal with an HWE (>=5.12) kernel, makedumpfile can sometimes fail with "__vtop4_x86_64: Can't get a valid pmd_pte."
* makedumpfile falls back to cp for the dump, resulting in extremely large vmcores. This can impact both collection and analysis due to lack of space for the resulting vmcore.
* This is fixed in upstream commit present in versions 1.7.0 and 1.7.1:
https:/
commit 646456862df8926
Author: Kazuhito Hagio <email address hidden>
Date: Wed May 26 14:31:26 2021 +0900
[PATCH] Increase SECTION_
* Required for kernel 5.12
Kernel commit 1f90a3477df3 ("mm: teach pfn_to_
ZONE_DEVICE section collisions") added a section flag
(SECTION_
some machines like this:
_
readmem: Can't convert a virtual address(
readmem: type_addr: 0, addr:ffffe2bdc2
_
create_
Increase SECTION_
been used until the change, so we can just increase the value.
Signed-off-by: Kazuhito Hagio <email address hidden>
[Test Plan]
* Confirm that makedumpfile works as expected by triggering a kdump.
* Confirm that the patched makedumpfile works as expected on a system known to experience the issue.
* Confirm that the patched makedumpfile is able to work with a cp-generated known affected vmcore to compress it. The unpatched version fails.
[Where problems could occur]
* This change could adversely affect the collection/
Changed in makedumpfile (Ubuntu): | |
assignee: | nobody → Kellen Renshaw (krenshaw) |
tags: | added: sts |
summary: |
- makedumpfile fails with __vtop4_x86_64: Can't get a valid pmd_pte. + makedumpfile falls back to cp with "__vtop4_x86_64: Can't get a valid + pmd_pte." |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Guilherme G. Piccoli (gpiccoli) wrote : | #1 |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Kellen Renshaw (krenshaw) wrote : | #2 |
Hi Guilherme!
I have been looking at the possibility of an SRU exception for makedumpfile, although I like your idea better (LTS N to LTS N+1).
How would we go about implementing that? I am willing to do the building/testing work, but would need guidance on how to do that and properly record/report the results.
Thanks!
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Thadeu Lima de Souza Cascardo (cascardo) wrote : | #3 |
There are two risks with that plan that we should overcome.
One is testing, such updates should not cause regressions. As of right now, the small testing that makedumpfile receives is not sufficient and gives a lot of false negatives. We should be testing that new kernels are still dumpable (and fix either kernel or makedumpfile when they are not). And test that new makedumpfile versions do not break dumping all the supported kernel versions (which, in my opinion is a little harder, and puts some burden on makedumpfile updates). Users do run outdated kernels and would expect dumps when they crash, so this is a bit of a challenge. We do not need to be perfect and test all kernels in all scenarios, but we definitively need to do better.
The second one is kernel support. It's not unusual that we release an Ubuntu version with a makedumpfile that cannot dump the GA kernel. So, even without considering HWE kernels, an LTS release may need a newer makedumpfile. One of the reasons is that as we don't test as we upload new kernels to the development series, we don't realize makedumpfile needs additional support for that new kernel. Sometimes, just having the latest released makedumpfile is sufficient. But it's too often the case that upstream makedumpfile is only able to catch up with latest kernel releases after a while.
Cascardo.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Guilherme G. Piccoli (gpiccoli) wrote : | #4 |
Thanks Kellen and Cascardo.
Kellen, nice that you're willing to work on that - this is a long standing problem and that work would be definitely appreciated by the Ubuntu community, be it free users or the Ubuntu Advantage customers!
Cascardo, about your two risks:
(a) Partially agree with that. I agree with the part of testing, definietly this is the big chunk of work here. But I disagree with the retrocompatibility claim: of course we need to enforce that, but it's not that difficult in the LTS->LTS+1 model. See, we have a small number of HWE kernel per LTS release, I guess 4 or 5 correct? We need to be sure the makedumpfile updates are compatible with them, and that's it.
If I'm talking Focal and some makedumpfile update (unintentionally) breaks dumps for kernel 4.19 or prior, why should we care if that's an unsupported scenario?
IMHO it's much better to ensure that every HWE kernel receives a proper functional makedumpfile update, instead of an overly cautious attitude with older/unsupported kernels.
(b) I agree here, but I guess the effort of SRU exception/ LTS->LTS+1 model will only make it easier. Imagine if when Ubuntu version X is released the upstream makedumpfile is not handling well the recent kernel version used by X - so we could either fix (or report) the makedumpfile issue quickly (especially due to part (a) above, the improved testing). Then, once it's fixed either by Canonical or community, this could quickly be integrated through a fast process, a version bump for makedumpfile for example.
In the end, I think testing is the key word here - the more serious and thorough tests makedumpfile has, the more confidence in such model we'd have. But hopefully with Kellen's effort this stops preventing a more proactive approach with makedumpfile from happening, by updating it before users report bugs (which has been happening since forever for this package).
Cheers!
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Thadeu Lima de Souza Cascardo (cascardo) wrote : | #5 |
Hi, Guilherme.
I think you misunderstood and we are in agreement. What I mean by the first point is that, on Focal, we need to support 5.4 and 5.15. I don't even think we need to support 5.8 and 5.11 any longer, though 5.13, as it will still be supported for a while needs supporting. But I also mean that we should support not only 5.4.0-113 and 5.4.0-114 (versions on -updates and -proposed), we should also support older 5.4 versions (one could argue all the versions that are still livepatchable at least). I don't think the risk of regressions is big here, but it exists. We ought to balance that risk and how much we can test.
About the second point, it's just an argument that we should not restrict ourselves to the upstream major makedumpfile version that is in LTS+1. Take focal, which will receive 5.15 from jammy. It may require a newer makedumpfile than the one in Jammy because the one in Jammy may not be sufficient to support 5.15.
Cascardo.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dann frazier (dannf) wrote : | #6 |
Thanks everyone for trying to tackle this long-standing issue. fwiw, here's my $0.02 no how we could proceed:
Someone should draft a special case page for makedumpfile:
https:/
I'm happy to review/provide feedback, but I'd rather someone who would be carrying out the plan drive it.
As others have mentioned, testing is the hard part, and we need to define what will be tested in the special case documentation. Since makedumpfile is really just a filter, I don't think we need to (or reasonably could) boot a bunch of systems in different configs and generate crashdumps for every new update. Rather, i think we could build a repository of representative, unfiltered, /proc/vmcore files that focal's existing makedumpfile can parse. Then we can just check that all of those files can still be parsed by the proposed makedumpfile. With some scripting and a multi-architecture cloud, this could be automated. In fact, if this vmcore repo were online, we could implement this an autopkgtest (w/ needs-internet set). But we should also do at least one end-to-end kdump, just to make sure the kdump-tools-
What is a representative sample? One of each of the current LTS and HWE kernels on amd64, arm64, ppc64el and s390x seems like an obvious start (or the subset of those that actually work today). I don't think the machine type is as important, VMs should be fine IMO. If we know of examples where different machines expose structures differently in a way that makedumpfile cares about, then perhaps add those as well. Once a new makedumpfile lands that adds support for a new HWE kernel, we should probably then update the repo w/ vmcore samples from that kernel, so we can make sure the next update doesn't regress that support (probably convenient to do when verifying the SRU, since I imagine we'd be testing that it works w/ the new HWE kernel then anyway).
It'd be good to note in the special case request that kdump-tools does fall back to a raw /proc/vmcore file cp if makedumpfile fails, which can mitigate regressions for a subset of users
- those with the necessary disk space and lack of time constraints.
While I agree that crash falls into the same category, I don't think it necessarily needs to happen at the same time. Obviously users running focal need to dump their vmcore using focal - bug for developers debugging a crash, I don't think it is to onerous to use a newer version of Ubuntu. Again, I'm no saying we *shouldn't* add crash to the special case, it just seems like a makedumpfile exception is significantly more important.
Finally, I don't think we need to commit to a frequency of backports, or a point at which they will stop. Rather we can just stick to agreeing on how it *can* be done when someone has the time/interest in doing it. Guillherme's LTS->LTS+1 scheme sounds like a reasonable pattern to shoot for, but if that doesn't happen every time, we're still improving the situation over the status quo.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Guilherme G. Piccoli (gpiccoli) wrote : | #7 |
Thanks a lot Cascardo and Dann - very good points.
Cascardo: I agree with you, I misunderstood and didn't consider the minor kernel releases. I think that Dann's idea of testing makes it much simpler though. Maybe kernel team could create the vmcore images, as part of the release process, for some random kernels (like generic + 3 cloud kernels randomly) and "dump" into this server. So, makedumpfile test infrastructure would then consume the vmcores and execute the test, checking against bugs/regressions. The test could be quite simple, just checking return value of makedumpfile and if the file created is in fact a compressed dump (the "file" tool could be used by that).
I agree with you as well Dann, makedumpfile is much more important than the crash tool and should have priority. Also, testing makedump is easier than checking the crash tool I guess heheh
Cheers!
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Kellen Renshaw (krenshaw) wrote : | #8 |
Thanks everyone for the awesome input and ideas! I will draft the special case exception and figure out some way to post it here for review.
Regarding testing, I really like Dann and Guilherme's idea for automated testing of vmcores.
Where would that infrastructure live and how would we maintain a repo (or just directory) of vmcores in a well-known location suitable for package testing?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Kellen Renshaw (krenshaw) wrote : | #9 |
During a discussion about the potential for autopkgtests it was brought up that a representative sample of unfiltered vmcores would be large (even compressed). This would impose bandwidth and disk space constraints on anyone attempting to run the autopkgtests locally.
That tends to make me lean toward setting up a library of cores with test infrastructure somewhere that is run as part of the SRU exception process, but not as an autopkgtest.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dann frazier (dannf) wrote : | #10 |
Yeah, I don't think an autopkgtest is a requirement. Having the test live within the package itself would make it more accessible, and autopkgtest was my first thought as to how to do that. But I don't see a "disable by default" flag, and I agree that huge downloads on every test run could be a problem. It would still be nice IMO if the test was integrated into the package though, even if it needs to run manually.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Launchpad Janitor (janitor) wrote : | #11 |
Status changed to 'Confirmed' because the bug affects multiple users.
Changed in makedumpfile (Ubuntu): | |
status: | New → Confirmed |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Kellen Renshaw (krenshaw) wrote : | #12 |
I have put up a draft SRU exception page at https:/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dann frazier (dannf) wrote : Re: [Bug 1970672] Re: makedumpfile falls back to cp with "__vtop4_x86_64: Can't get a valid pmd_pte." | #13 |
On Fri, Sep 23, 2022 at 11:15 AM Kellen Renshaw
<email address hidden> wrote:
>
> I have put up a draft SRU exception page at
> https:/
> I am working on finding an appropriate place for test vmcores to be
> stored.
Awesome! Thanks for starting this. I made some edits already, but
here's some other things I'd recommend:
- Provide links under resources (it's a web page!)
- be more precise about what will be tested, ideally in a checklist
form. Make it clear enough that you could give the list to another
developer and they would run the same tests and get the same results.
- Testing other architectures should not be optional IMO. makedumpfile
has architecture specific knowledge. You could easily break one w/o
noticing on another.
- Mention the plan to filter a pool of saved dump files as a
regression test. That is key IMO.
- If you have the regression testing pool, I don't see any reason to
have to do the full kdump process everywhere. Just do that once (one
kernel, one arch) to make sure the kdump-tools<
interface has been preserved. Testing the full crash dump process is
painful, but makedumpfile really is just used as a filter, and we can
test that easily.
-dann
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Kellen Renshaw (krenshaw) wrote : | #14 |
Thanks Dann!
Oof, the links didn't make it from the draft document, fixed now.
If I understand your previous comment correctly, a better variant of this testing procedure would be a regression testing checklist against a library (linked to in the page) of dump files and a single manual test of the kdump<>makedumpfile functionality?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dann frazier (dannf) wrote : | #15 |
On Wed, Sep 28, 2022 at 12:41 PM Kellen Renshaw
<email address hidden> wrote:
>
> Thanks Dann!
>
> Oof, the links didn't make it from the draft document, fixed now.
>
> If I understand your previous comment correctly, a better variant of
> this testing procedure would be a regression testing checklist against a
> library (linked to in the page) of dump files and a single manual test
> of the kdump<>makedumpfile functionality?
As someone who in no way speaks for the SRU team, yes - that is what
I'd suggest.
-dann
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Heather Lemon (hypothetical-lemon) wrote : | #16 |
Reposting this here as well https:/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Heather Lemon (hypothetical-lemon) wrote : | #17 |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Heather Lemon (hypothetical-lemon) wrote : | #18 |
Changed in makedumpfile (Ubuntu Focal): | |
assignee: | nobody → Heather Lemon (hypothetical-lemon) |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in makedumpfile (Ubuntu): | |
importance: | Undecided → Medium |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ubuntu Foundations Team Bug Bot (crichton) wrote : | #19 |
The attachment "lp1970672-
[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]
tags: | added: patch |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Dimitri John Ledkov (xnox) wrote : | #20 |
$ dput ubuntu makedumpfile_
D: Setting host argument.
Checking signature on .changes
gpg: /tmp/makedumpfi
Checking signature on .dsc
gpg: /tmp/makedumpfi
Uploading to ubuntu (via ftp to upload.ubuntu.com):
Uploading makedumpfile_
Uploading makedumpfile_
Uploading makedumpfile_
Uploading makedumpfile_
Successfully uploaded packages.
Changed in makedumpfile (Ubuntu): | |
status: | Confirmed → Fix Released |
Changed in makedumpfile (Ubuntu Focal): | |
status: | Confirmed → In Progress |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Łukasz Zemczak (sil2100) wrote : Please test proposed package | #21 |
Hello Kellen, or anyone else affected,
Accepted makedumpfile into focal-proposed. The package will build now and be available at https:/
Please help us by testing this new package. See https:/
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-
Further information regarding the verification process can be found at https:/
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.
Changed in makedumpfile (Ubuntu Focal): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed verification-needed-focal |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Heather Lemon (hypothetical-lemon) wrote : | #22 |
### VERIFICATION-FAILED FOCAL-PROPOSED ###
makedumpfile original version: 1:1.6.7-1ubuntu2.4
makedumpfile proposed version: 1:1.6.7-1ubuntu2.5
kernel version: 5.15.0-89-generic hwe
sudo apt install -y linux-crashdump
sudo vim /etc/default/
# Add the line at the top of the file below USE_KDUMP=1
LOAD_KEXEC=true
# Uncomment the makedumpfile line and change 31 to 32
MAKEDUMP_ARGS="-c -d 32"
exit vim
sudo vim /etc/default/
change 192 to either 256M or 512M
sudo sysctl -w kernel.sysrq=1
sudo apt-get update-grub
sudo reboot
sudo su
kdump-config show
Needs to look similar to:
root@focal:
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x73000000
/var/
kdump initrd:
/var/
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-
*most importantly the addr is not zero and the state is: ready to kdump.
# trigger crash
echo c > /proc/sysrq-trigger
... wait for reboot and login
cd /var/crash/
There is a folder with a datetimestamp of the crash, inside the folder is the vmcore and dmesg files
There is also a file called linux-image*.crash in /var/crash/
*Note: In bionic we get the cp message
[ 6.946187] kdump-tools[622]: Starting kdump-tools: * running makedumpfile -c -d 32 /proc/vmcore /var/crash/
[ 6.959932] kdump-tools[622]: Dump_level(32) is invalid.
[ 6.964316] kdump-tools[622]: makedumpfile Failed.
[ 6.976231] kdump-tools[622]: * kdump-tools: makedumpfile failed, falling back to 'cp'
[ 25.084729] kdump-tools[622]: * kdump-tools: saved vmcore in /var/crash/
[ 26.355039] kdump-tools[622]: * running makedumpfile --dump-dmesg /proc/vmcore /var/crash/
[ 26.436513] kdump-tools[622]: The dmesg log is saved to /var/crash/
[ 26.443208] kdump-tools[622]: makedumpfile Completed.
[ 26.449066] kdump-tools[622]: * kdump-tools: saved dmesg content in /var/crash/
[ 26.463950] kdump-tools[622]: Wed, 22 Nov 2023 14:22:50 +0000
[ 26.490318] kdump-tools[622]: Rebooting.
*Note: In Focal we don't
Nov 28 13:35:54 focal kdump-tools[604]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: n: failed to create symbolic link '/va: No such file or directory
Nov 28 13:35:54 focal kdump-tools[713]: kdump-tools: Generating /var/lib/
Nov 28 13:35:57 focal kdump-tools[667]: * Creating syml...
tags: |
added: verification-failed-focal removed: verification-needed-focal |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Fabio Augusto Miranda Martins (fabio.martins) wrote : | #23 |
I've tested makedumpfile from -proposed on Focal and it looks good to me.
Using a vmcore file with 2TB as an input:
- Original makedumpfile 1.6.7-1ubuntu2.4 fails:
ubuntu@
makedumpfile:
Installed: 1:1.6.7-1ubuntu2.4
Candidate: 1:1.6.7-1ubuntu2.4
Version table:
*** 1:1.6.7-1ubuntu2.4 500
500 http://
100 /var/lib/
1:
500 http://
ubuntu@
The kernel version is not supported.
The makedumpfile operation may be incomplete.
Checking for memory holes : [100.0 %] / __vtop4_x86_64: Can't get a valid pmd_pte.
readmem: Can't convert a virtual address(
readmem: type_addr: 0, addr:ffffecff81
__exclude_
create_2nd_bitmap: Can't exclude unnecessary pages.
makedumpfile Failed.
- Makedumpfile 1.6.7-1ubuntu2.5 from proposed works:
ubuntu@
makedumpfile:
Installed: 1:1.6.7-1ubuntu2.5
Candidate: 1:1.6.7-1ubuntu2.5
Version table:
*** 1:1.6.7-1ubuntu2.5 500
500 http://
100 /var/lib/
1:
500 http://
1:
500 http://
ubuntu@
The kernel version is not supported.
The makedumpfile operation may be incomplete.
Copying data : [100.0 %] - eta: 0s
The dumpfile is saved to ./dump-
makedumpfile Completed.
It reduced the dump file from 2TB down to 4.5G:
ubuntu@
-r-------- 1 ubuntu ubuntu 2.0T Apr 21 2022 vmcore.202204202351
ubuntu@
-rw------- 1 ubuntu ubuntu 4.5G Dec 12 14:23 dump-incomplete
The reason for having a vmcore file with the size of the installed RAM in the comment reported by Heather, is that you are forcing makedumpfile to fail, by providing "-c -d 32" (which is a level that doesn't exist, as the max is 31) or moving the makedumpfile binary away, so kdump fails over to cp, which hence will produce the vmcore file with the size of the installed RAM.
Let me know if this is enough to have focal verification concluded.
tags: |
added: verification-done-focal removed: verification-failed-focal verification-needed |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Heather Lemon (hypothetical-lemon) wrote : | #24 |
Ah okay that makes sense, thanks for re-testing. What you've done looks good to me.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Chris Halse Rogers (raof) wrote : | #25 |
So, the test plan was:
```
[Test Plan]
* Confirm that makedumpfile works as expected by triggering a kdump.
* Confirm that the patched makedumpfile works as expected on a system known to experience the issue.
* Confirm that the patched makedumpfile is able to work with a cp-generated known affected vmcore to compress it. The unpatched version fails.
```
I'm not familiar with this area, but as far as I can tell only the 3rd step has been tested here? Am I misunderstanding the comments, or have we missed some of the test plan?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Fabio Augusto Miranda Martins (fabio.martins) wrote : | #26 |
Hi Chris,
You're correct, I'm sorry. My test on comment #23 is the 3rd item you listed.
Let me work on 1 and 2 and I'll get back here.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Fabio Augusto Miranda Martins (fabio.martins) wrote : | #27 |
For item 2:
* Confirm that the patched makedumpfile works as expected on a system known to experience the issue.
Unfortunately I'm no longer able to reproduce the original issue.
Even running on the same hardware where this was originally noticed, with the same kernel version (5.13.0-
[ 53.223512] kdump-tools[693]: Starting kdump-tools:
[ 53.623944] kdump-tools[702]: * running makedumpfile -c -d 31 /proc/vmcore /var/crash/
Copying data : [ 196.965120] reboot: Restarting system
[ 22.0 %] |
Unfortunately I don't have the information and I don't have access to the original system to check what version of makedumpfile it was using back then, so I could test the exact same makedumpfile+kernel versions.
I also tested kernel 5.13.0-1027-oracle + makedumpfile 1:1.6.7-1ubuntu2 from focal/main, and in this combinarion, makedumpfile fails with a similar, but slightly different error, then falls back to cp:
[ 53.721130] kdump-tools[690]: Starting kdump-tools:
[ 54.121624] kdump-tools[699]: * running makedumpfile -c -d 31 /proc/vmcore /var/crash/
[ 54.249624] kdump-tools[719]: get_mm_sparsemem: Can't get the address of mem_section.
[ 54.345410] kdump-tools[719]: The kernel version is not supported.
[ 54.425405] kdump-tools[719]: The makedumpfile operation may be incomplete.
[ 54.517391] kdump-tools[719]: makedumpfile Failed.
[ 54.577916] kdump-tools[699]: * kdump-tools: makedumpfile failed, falling back to 'cp'
However, using the latest makedumpfile from focal-updates/main (1:1.6.
Due to this reason, I can't conclude the item 2.
I'll work now on 1.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Fabio Augusto Miranda Martins (fabio.martins) wrote : | #28 |
For item 1:
* Confirm that makedumpfile works as expected by triggering a kdump.
I can confirm that makedumpfile 1:1.6.7-1ubuntu2.5 from focal-proposed/main worked well when I triggered a dump in a system:
ubuntu@
Static hostname: fabio-small-
Icon name: computer-vm
Chassis: vm
Machine ID: dee0adfb9aa5424
Boot ID: adba6ba3977f4c7
Virtualization: oracle
Operating System: Ubuntu 20.04.6 LTS
Kernel: Linux 5.15.0-1049-oracle
Architecture: x86-64
ubuntu@
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x2c000000
0xfd7f000000
/boot/
kdump initrd:
/boot/
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-
ubuntu@
Desired=
| Status=
|/ Err?=(none)
||/ Name Version Architecture Description
+++-===
ii makedumpfile 1:1.6.7-1ubuntu2.5 amd64 VMcore extraction tool
ubuntu@
makedumpfile:
Installed: 1:1.6.7-1ubuntu2.5
Candidate: 1:1.6.7-1ubuntu2.5
Version table:
*** 1:1.6.7-1ubuntu2.5 500
500 http://
100 /var/lib/
1:
500 http://
1:
500 http://
Output showing that it completed well:
[ 54.490112] kdump-tools[676]: Starting kdump-tools:
[ 54.876357] kdump-tools[686]: * running makedumpfile -c -d 31 /proc/vmcore /var/crash/
Checking for memory holes : [100.0 %] \ [ 204.391465] reboot: Restarting system
And when I look at the crash, it's properly compressed (system had 1TB of RAM):
ubuntu@
total 2.3G
-rw------- 1 root root 126K Dec 15 15:26 dmesg.202312151524
-rw------- 1 root root 2.3G Dec 15 15:26 dump.202312151524
Regards,
Fabio Martins
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Chris Halse Rogers (raof) wrote : | #29 |
Ok. My understanding of this is that when (3) fails it fails for the same reason that (2) would. We've verified that making a dumpfile works in general, and definitely fixed (3), so I'm going to release this now.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Launchpad Janitor (janitor) wrote : | #30 |
This bug was fixed in the package makedumpfile - 1:1.6.7-1ubuntu2.5
---------------
makedumpfile (1:1.6.
* makedumpfile falls back to cp with 5.12 kernel (LP: #1970672)
- can also fail with __vtop4_x86_64: Can't get a valid pmd_pte.
- d/p/lp1970672-
-- Heather Lemon <email address hidden> Tue, 21 Nov 2023 15:19:22 +0000
Changed in makedumpfile (Ubuntu Focal): | |
status: | Fix Committed → Fix Released |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Chris Halse Rogers (raof) wrote : Update Released | #31 |
The verification of the Stable Release Update for makedumpfile has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.
Hi Kellen, thanks a lot for reporting and fixing that!
I'd like to take the opportunity to discuss something related: no matter how many bugs we fix in makedumpfile / crash, more will come as kernel version bumps. Kernel has no stable ABI, so kernel developers can "break" compatibility with such tools, although makedumpfile maintainer (and crash's as well!) are really great in keep up with that and release proactive fixes even before the kernel change is merged.
But the problem is: in Ubuntu ecosystem, despite we have the HWE concept for kernel, these packages are not part of kernel HWE upgrades; hence, they get "stuck" and subject to bugs when kernel HWE is released. It happens all the time and will continue happening...
We had discussions in the past (and I'm hereby CCing the interested parties: DannF, Dan Streetman, Heitor and Cascardo) about sync'ing makedumpfile and crash with kernel HWE upgrades. So, that might be a good opportunity for doing it.
The idea was more or less like this: update makedump/crash on Release to make it sync'ed with Release +1 until the next LTS. So, in the end, we'll have LTS version == LTS +1 and then, we stop upgrading/syncing these packages. And the cycle restarts for LTS+1, up to the release of LTS+2.
Hopefully this plan (or something similar) eventually is followed, I bet all users/customers would be glad to not face makedump/crash bugs due to kernel upgrades anymore!
Cheers, and thanks for the attention =D