systemd-coredump handler does not forward the crash to the container

[Ravi Kant Sharma]

Steps to reproduce:
1. install systemd-coredump on the host
2. open a shell on a noble container
2.1 machinectl pull-raw "http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img" noble
2.2 machinectl start noble
2.3 machinectl shell noble
3. install chaos-marmosets inside the container (# apt install chaos-marmosets)
4. generate a crash inside the container (# divide-by-zero)

Observed behaviour: The .crash file is present on the host /var/crash
Expected behaviour: The .crash file should be present inside the container /var/crash

Note: Here is how you can enable Host Network on the container https://wiki.archlinux.org/title/systemd-nspawn#Use_host_networking

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: apport 2.28.1-0ubuntu2
ProcVersionSignature: Ubuntu 6.8.0-22.22-generic 6.8.1
Uname: Linux 6.8.0-22-generic x86_64
NonfreeKernelModules: zfs
ApportLog:
 INFO: apport (pid 126611) 2024-04-24 16:34:39,226: executable: /usr/bin/divide-by-zero (command line "divide-by-zero")
 INFO: apport (pid 126611) 2024-04-24 16:34:39,664: wrote report /var/crash/_usr_bin_divide-by-zero.1000.crash
 INFO: apport (pid 127435) 2024-04-24 16:41:16,322: executable: /usr/bin/divide-by-zero (command line "divide-by-zero")
 INFO: apport (pid 127435) 2024-04-24 16:41:16,758: wrote report /var/crash/_usr_bin_divide-by-zero.297664512.crash
ApportVersion: 2.28.1-0ubuntu2
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Wed Apr 24 17:03:02 2024
InstallationDate: Installed on 2024-04-18 (6 days ago)
InstallationMedia: Ubuntu 24.04 LTS "Noble Numbat" - Beta amd64 (20240418)
PackageArchitecture: all
ProcEnviron:
 LANG=en_US.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=xterm-256color
 XDG_RUNTIME_DIR=<set>
RebootRequiredPkgs: Error: path contained symlinks.
SourcePackage: apport
UpgradeStatus: No upgrade log present (probably fresh install)

[Nick Rosbrook (enr0n)]

Did you verify that systemd-coredump is installed in the container?

[Nick Rosbrook (enr0n)]

root@selected-slug:~# machinectl pull-raw "http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img" noble
Enqueued transfer job 1. Press C-c to continue download in background.
Pulling 'http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img', saving as 'noble'.
Downloading 453.0M for http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img.
HTTP request to http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.roothash.p7s failed with code 404.
Root hash signature file could not be retrieved, proceeding without.
HTTP request to http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.verity failed with code 404.
Verity integrity file could not be retrieved, proceeding without.
HTTP request to http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.nspawn failed with code 404.
Settings file could not be retrieved, proceeding without.
Downloading 833B for http://cloud-images.ubuntu.com/noble/current/SHA256SUMS.gpg.
Acquired 833B.
Download of http://cloud-images.ubuntu.com/noble/current/SHA256SUMS.gpg complete.
HTTP request to http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.roothash failed with code 404.
Root hash file could not be retrieved, proceeding without.
Downloading 6.9K for http://cloud-images.ubuntu.com/noble/current/SHA256SUMS.
Got 37% of http://cloud-images.ubuntu.com/noble/current/SHA256SUMS.
Acquired 6.9K.
Download of http://cloud-images.ubuntu.com/noble/current/SHA256SUMS complete.
Got 1% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 2min 13s left at 3.3M/s.
Got 5% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 41s left at 10.4M/s.
Got 11% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 26s left at 14.9M/s.
Got 16% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 21s left at 17.5M/s.
Got 22% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 18s left at 19.0M/s.
Got 28% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 16s left at 20.1M/s.
Got 33% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 14s left at 20.6M/s.
Got 39% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 13s left at 21.1M/s.
Got 44% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 11s left at 21.3M/s.
Got 50% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 10s left at 21.8M/s.
Got 53% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 9s left at 21.3M/s.
Got 58% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 8s left at 21.4M/s.
Got 64% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 7s left at 21.7M/s.
Got 69% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 6s left at 21.9M/s.
Got 75% of http://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img. 4s left at 22.2M/...

[Benjamin Drung (bdrung)]

Nick, it works if you have systemd-coredump installed in the container.

If you do not have systemd-coredump installed in the container, apport on the host should do the forwarding.

[Ravi Kant Sharma (ravi-sharma)]

>Did you verify that systemd-coredump is installed in the container?

It works when systemd-coredump is installed inside the container. This report is for the default noble container. My understanding is that in no case should the .crash file be on the host when a crash occurs inside a container. I guess apport should forward the crash to the container in this case.

[Nick Rosbrook (enr0n)]

Okay. The title should be changed then -- systemd-coredump on the host attempts to forward the crash to the container, but finds that systemd-coredump is not present in the container, so it falls back to keeping it on the host.

[Benjamin Drung (bdrung)]

Apport cannot do anything useful with these crash reports, because the information about the crashed process does not contain the process number seen from inside the container and because the crashed process is already gone when Apport gets the crash from systemd-coredump.

So let Apport just ignore crash reports from systemd-coredump if the crashes happened inside containers: https://github.com/canonical/apport/pull/326