systemctl daemon-reexec forgets running services and starts everything new

[Erik Wasser]

[Impact]

Depending on the contents of /proc/cmdline, when systemd is re-executed with systemctl daemon-reexec, the --deserialize flag may be ignored because it was added after the other arguments. For example, if /proc/cmdline contains ---, then the re-exec cmdline might look like:

$ cat /proc/1/cmdline | tr '\0' '\n'
/lib/systemd/systemd
---
splash
--system
--deserialize
54

This causes systemd not to process the --deserialize 54 argument, causing it to start with a fresh state. This can cause all kinds of problems, and one easy symptom to see is many lines in the journal like:

"$service.service: Found left-over process $pid ($service) in control group while starting unit. Ignoring."

[Test Plan]

1. (Only needed if your test system is not already affected) Edit the kernel command line to contain '---' at the end, which would trigger the bug. This can be done by appending '---' to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, running update-grub, and then rebooting.

2. After enabling -proposed, install systemd:

$ apt install systemd -y

3. Check that the systemd.postinst script skipped the daemon-reexec call, and instead indicated a reboot is required:

$ grep -Fsx systemd /run/reboot-required.pkgs
systemd

4. Reboot.

5. Try to re-exec systemd, and check that there are not tons of "left-over process" log messages:

$ systemctl daemon-reexec
$ journalctl --grep "Found left-over process" -b 0

6. Also confirm that the ordering of /proc/1/cmdline is correct, i.e. that --deserialize $fd comes before args from /proc/cmdline:

$ cat /proc/1/cmdline | tr '\0' '\n'

[Where problems could occur]

There are two changes for this bug. First is the patch against systemd itself, which changes the ordering of arguments on the systemd commandline. This change simply makes it so that systemd's own arguments are always put first on it's re-exec commandline, and that anything from /proc/cmdline is appended after. Any regressions caused by this would also be seen in systemctl daemon-reexec invocations.

The second change is in systemd.postinst, which skips the systemctl daemon-reexec call when upgrading from versions of systemd that could hit this bug. Regressions caused by this would be seen during package upgrades.

[Original Description]

# Our problem #

During a regular update of our container environment, `systemd` (and the related packages libpam-systemd, libsystemd0, libudev1, systemd-sysv and udev) were updated from `249.11-0ubuntu3.6` to `249.11-0ubuntu3.7`. We're talking only about Ubuntu 22.04. Our Ubuntu 20.04 is working fine with `systemctl daemon-reexec`.

In my opinion, the update was not the problem because we've tried downgrading and tried these versions: (current) `249.11-0ubuntu3.7`, `249.11-0ubuntu3.6`, `249.11-0ubuntu3.4` and `249.11-0ubuntu3.3`. The symptoms were the same.

# Symptoms #

The `/var/lib/dpkg/info/systemd.postinst` executes a `systemctl daemon-reexec` and that ended in a disaster. It seems that `systemd` is forgetting all it started children and tries to start nearly every configured service again. Naturally, the old services are still running, and the ports can't be opened twice and `systemd` won't give up. Here are some(!) of the logfiles:

Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Starting Create Volatile Files and Directories...
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: systemd-udevd.service: Found left-over process 130 (systemd-udevd) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: systemd-udevd.service: Found left-over process 31475 (systemd-udevd) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: systemd-udevd.service: Found left-over process 31476 (systemd-udevd) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.

And...

Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Reached target System Initialization.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily apt download activities.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily apt upgrade and clean activities.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily dpkg database backup timer.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Periodic ext4 Online Metadata Check for All Filesystems.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Condition check resulted in Discard unused blocks once a week being skipped.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily rotation of log files.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily man-db regeneration.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Message of the Day.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Clean PHP session files every 30 mins.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Update the plocate database daily.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Daily Cleanup of Temporary Directories.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Reached target Basic System.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: System is tainted: cgroupsv1
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Reached target Timer Units.

And...

Mar 31 12:51:39 FQDN_REDACTED systemd[1]: atd.service: Found left-over process 206 (atd) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Starting Deferred execution scheduler...
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: cron.service: Found left-over process 164 (cron) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started Regular background program processing daemon.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: dbus.service: Found left-over process 177 (dbus-daemon) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Started D-Bus System Message Bus.

And...

Mar 31 12:51:39 FQDN_REDACTED systemd[1]: rsyslog.service: Found left-over process 204 (rsyslogd) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: atd.service: Failed with result 'exit-code'.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: atd.service: Unit process 206 (atd) remains running after unit stopped.

Mar 31 12:51:39 FQDN_REDACTED systemd[1]: apache2.service: Found left-over process 382 (apache2) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: apache2.service: Found left-over process 392 (apache2) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: apache2.service: Found left-over process 397 (apache2) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: apache2.service: Found left-over process 3052 (apache2) in control group while starting unit. Ignoring.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Starting The Apache HTTP Server...
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: Stopped Deferred execution scheduler.
Mar 31 12:51:39 FQDN_REDACTED systemd[1]: atd.service: Found left-over process 206 (atd) in control group while starting unit. Ignoring.

And...

Mar 31 12:51:40 FQDN_REDACTED sshd[31772]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.

And...

Mar 31 12:52:06 FQDN_REDACTED systemd[1]: Started The Salt Minion.
Mar 31 12:52:06 FQDN_REDACTED salt-minion[32339]: The Salt Minion is shutdown.
Mar 31 12:52:11 FQDN_REDACTED systemd[1]: salt-minion.service: Main process exited, code=exited, status=1/FAILURE
Mar 31 12:52:11 FQDN_REDACTED systemd[1]: salt-minion.service: Failed with result 'exit-code'.
Mar 31 12:52:11 FQDN_REDACTED systemd[1]: salt-minion.service: Unit process 2808 (/opt/saltstack/) remains running after unit stopped.
Mar 31 12:52:11 FQDN_REDACTED systemd[1]: salt-minion.service: Unit process 2848 (/opt/saltstack/) remains running after unit stopped.

Other internal `systemd` process were started again:

root 1 0.0 0.1 101204 12444 ? Ss 10:19 0:03 /lib/systemd/systemd -z --system --deserialize 16
root 75 0.0 0.1 31440 13484 ? Ss 10:19 0:00 /lib/systemd/systemd-journald
systemd+ 159 0.0 0.0 16124 8004 ? Ss 10:19 0:00 /lib/systemd/systemd-networkd
message+ 177 0.0 0.0 8252 4440 ? Ss 10:19 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root 205 0.0 0.0 14908 6464 ? Ss 10:19 0:00 /lib/systemd/systemd-logind
systemd+ 223 0.0 0.1 25268 12592 ? Ss 10:19 0:00 /lib/systemd/systemd-resolved
root 31424 0.0 0.1 31424 13636 ? Ss 12:51 0:00 /lib/systemd/systemd-journald
systemd+ 31636 0.0 0.0 16124 6588 ? Ss 12:51 0:00 /lib/systemd/systemd-networkd
message+ 31639 0.0 0.0 8124 3804 ? Ss 12:51 0:00 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root 31682 0.0 0.0 14908 6480 ? Ss 12:51 0:00 /lib/systemd/systemd-logind
systemd+ 31686 0.0 0.1 25268 12580 ? Ss 12:51 0:00 /lib/systemd/systemd-resolved
root 32087 0.0 0.0 21436 5252 ? Ss 12:51 0:00 /lib/systemd/systemd-udevd

You can either kill all the old processes and restart them, and then everything is fine. Or you can reboot the container. Besides that `systemctl daemon-reexec` the `systemd` version is running fine. `systemctl daemon-reload` is working like a charme.

# Normal case #

In the normal case a `systemctl daemon-reexec` just prints only a few lines:

Mar 31 14:21:58 FQDN_REDACTED systemd[1]: Reexecuting.
Mar 31 14:21:58 FQDN_REDACTED systemd[1]: systemd 249.11-0ubuntu3.7 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Mar 31 14:21:58 FQDN_REDACTED systemd[1]: Detected architecture x86-64.

# Testcase #

Doing a `systemctl daemon-restart` and `ssh localhost` shows the problem. `systemctl` removes the directory `/run/sshd` during the reexec and `ssh` will refuse further connects because the directory is missing.

$ systemctl daemon-restart
$ ssh root@localhost
kex_exchange_identification: read: Connection reset by peer
Connection reset by 127.0.0.1 port 22
$

Killing the old instance of SSH and restarting it will work.

# Some details to the hardware #

Our metal runs OpenVZ/Virtuozzo with this kernel (without any problems):

> Linux FQDN_REDACTED 3.10.0-1127.18.2.vz7.163.46 #1 SMP Fri Nov 20 21:47:55 MSK 2020 x86_64 x86_64 x86_64 GNU/Linux

The container with the `systemctl daemon-reexec` problem reports the following kernel:

Linux FQDN_REDACTED 5.4.0 #1 SMP Thu Apr 22 16:18:59 MSK 2021 x86_64 x86_64 x86_64 GNU/Linux

# Upshot #

* Can somebody help me with this issue?
* Why is `systemctl` losing its internal state about the running processes/services?
* Why is `systemctl` restarting everything?

[Nick Rosbrook (enr0n)]

To clarify, you are saying that this issue occurs any time you run `systemctl daemon-reexec`, right? It is not isolated to that command being run during a package upgrade?

I am not familiar with OpenVZ, but I am confused by the kernel situation. Are you saying that the containers which are running Ubuntu (and where you are seeing this problem) are running different kernels than the host?

Like I said, I am not familiar with OpenVZ, but from my understanding of other container environments like LXD, I would expect the containers to share the host's kerenl (and therefore report the same uname -a output).

[Erik Wasser (ewasser)]

Clarification: Yes, we've isolated the behavior down to the call of `system daemon-reexec` from the `/var/lib/dpkg/info/systemd.postinst` file.

But every call of `system daemon-reexec` will trigger this "bug", if you have installed the packages in version `249.11-0ubuntu3.4`.

The host machine and the container are running the same kernel. But you can fake the kernel version in the container.

So, the host machine reports the real version of the kernel:

> Linux FQDN_REDACTED 3.10.0-1127.18.2.vz7.163.46 #1 SMP Fri Nov 20 21:47:55 MSK 2020 x86_64 x86_64 x86_64 GNU/Linux

And the container reports this:

> Linux FQDN_REDACTED 5.4.0 #1 SMP Thu Apr 22 16:18:59 MSK 2021 x86_64 x86_64 x86_64 GNU/Linux

This was implemented because some applications in the container are checking for a newer kernel version, so you have to fake it.

[Nick Rosbrook (enr0n)]

> This was implemented because some applications in the container are checking for a newer kernel version, so you have to fake it.

Hm, this sounds like it would create plenty of bugs in userspace; presumably applications have a good reason for checking the kernel version, i.e. requiring certain functionality.

Can you please enable debug logging in systemd, and then provide some logs:

# sed -i 's/^#LogLevel=info/LogLevel=debug' /etc/systemd/system.conf
# reboot
# journalctl -t systemd -b 0 > logs.txt

Also, in a container *and* on the host, can you please report the output of this script:

```
#!/bin/bash

if [[ "$(stat -fc %T /sys/fs/cgroup/)" == "cgroup2fs" ]]; then
    echo "cgroupsv2"
elif [[ -e /sys/fs/cgroup/unified ]]; then
    echo "hybrid"
else
    echo "cgroupsv1"
fi
```

Edit: I had a typo in the script. The final "else" should print "cgroupsv1", not "cgroupsv2".

[Erik Wasser (ewasser)]

@enr0n: Thanks for your help.

I've deleted my old comment because I messed up the output of the container.

Here are the outputs of your test script. Output of the host:

hostsystem ~ # bash /tmp/test.sh
cgroupsv1

Output of the container:

container ~ $ bash /tmp/test.sh
hybrid

So, there is a mismatch of the cgroups versions. What does that mean?

I've attached the output of the `journalctl -t systemd -b 0`. Do you need an additional output after I've executed the command `systemctl daemon-reexec`?

Note: Executing the test scripts on a working(!) container with Ubuntu 20.04 and systemd 245.4-4ubuntu3.20 outputs the following:

hostsystem: cgroupsv1
container: hybrid

[Tim Ritberg (xpert-reactos)]

this problem is annoying...

[Nick Rosbrook (enr0n)]

Erik - Thanks for the output. Unfortunately I have not had time to dive deeper into this yet. In the mean time, would it be possible for you to test a container running 23.04? It would be helpful to know if it's still an issue in systemd v252.

[Nick Rosbrook (enr0n)]

Tim - Are you also experiencing this issue in an OpenVZ environment?

[Tim Ritberg (xpert-reactos)]

Nick - yes. It is a rented linux box.

[Erik Wasser (ewasser)]

Nike - Unfortunately, I don't have the resources to put a new image in our environment. But I've updated systemd (and all related) libraries to `252.5-2ubuntu3` and it's still "broken": `systemctl daemon-reexec` forgets all the old processes and restarts everything.

[Simon McVittie (smcv)]

I can reproduce what I think is the same problem on the original Ubuntu 22.04 live image ubuntu-22.04-desktop-amd64.iso (sha256: b85286d9855f549ed9895763519f6a295a7698fb9c5c5345811b3eefadfb6f07):

* boot a virtual machine with the live image in the virtual CD-ROM drive, and no hard drive
* wait for it to boot
* "Try Ubuntu"
* open a gnome-terminal
* in the terminal: sudo systemctl daemon-reexec

I'm retrying it in the equivalent 22.04.2 image.

[Simon McVittie (smcv)]

The latest Ubuntu 22.04.2 desktop live image (ubuntu-22.04.2-desktop-amd64.iso, sha256 b98dac940a82b110e6265ca78d1320f1f7103861e922aa1a54e4202686e9bbd3) exhibits what appears to be the same problem, which should make this very easy for a developer to reproduce.

[Nick Rosbrook (enr0n)]

Simon - are you also running in OpenVZ or is this a different virtualization environment?

[Tim Wagner (twagnersys11)]

Seeing the boot log from Simon McVittie (smcv), this is QEMU kvm machine.

-> May 09 14:32:15 ubuntu kernel: DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014

Also a colleague and me could replicate this behavior (With the same virtualization platform, same disk image...).

[Nick Rosbrook (enr0n)]

Sorry for the delay in response. I have not had sufficient time to investigate this further. Since it sounds like this can be observed on recent systemd releases, and there are no Ubuntu specific patches that stand out to me that might be causing this, please open an upstream bug report so that the issue can be investigated further.

[Kubìc Grünfeld (kubic.gruenfeld)]

I've created an upstream issue with some interesting findings so far: https://github.com/systemd/systemd/issues/28184

[Nick Rosbrook (enr0n)]

It looks like the upstream issue has been fixed, so we can look into getting this fixed in the next Jammy SRU.

[Christian Ehrhardt  (paelzer)]

We've debugged a different issue which might eventually be the very same root cause - at least superficially it looks similar/related.
But OTOH we do not have enough evidence yet to mark bug 2004551 a real duplicate of this yet.

@Nick once you have a test build or something in proposed with this fix would you give us a ping to retest in regard to bug 2004551 as well please?

[Nick Rosbrook (enr0n)]

Hi Christian,

Yes, I will keep you posted. Thanks for finding this connection. I have asked a follow-up on the other bug to help determine if this bug is also the root cause.

[Sergio Durigan Junior (sergiodj)]

Hm, I think we do have enough evidence to link the two problems.

Nick pinged me to let me know that he'd prepared a PPA with the backported upstream patch, and asked if I could test it on horsea. I did that, and verified that the new systemd package fixes the problem:

ubuntu@node-horsea:~$ sudo systemctl status ssh
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-08-04 18:47:55 UTC; 2min 35s ago
       Docs: man:sshd(8)
             man:sshd_config(5)
    Process: 884 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
   Main PID: 901 (sshd)
      Tasks: 1 (limit: 38215)
     Memory: 7.7M
        CPU: 84ms
     CGroup: /system.slice/ssh.service
             └─901 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups"

Aug 04 18:47:55 node-horsea systemd[1]: Starting OpenBSD Secure Shell server...
Aug 04 18:47:55 node-horsea sshd[901]: Server listening on 0.0.0.0 port 22.
Aug 04 18:47:55 node-horsea sshd[901]: Server listening on :: port 22.
Aug 04 18:47:55 node-horsea systemd[1]: Started OpenBSD Secure Shell server.
Aug 04 18:50:25 node-horsea sshd[1167]: Accepted publickey for ubuntu from 10.172.68.194 port 42610 ssh2: RSA SHA256:7JLFXGJ/lkuhcSu8/OF8mHjZ8wMD1zEfNSuNGM5u2Lk
Aug 04 18:50:25 node-horsea sshd[1167]: pam_unix(sshd:session): session opened for user ubuntu(uid=1000) by (uid=0)
ubuntu@node-horsea:~$ sudo systemctl daemon-reexec
ubuntu@node-horsea:~$ sudo systemctl status ssh
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-08-04 18:47:55 UTC; 2min 41s ago
       Docs: man:sshd(8)
             man:sshd_config(5)
   Main PID: 901 (sshd)
      Tasks: 1 (limit: 38215)
     Memory: 7.6M
        CPU: 84ms
     CGroup: /system.slice/ssh.service
             └─901 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups"

Aug 04 18:47:55 node-horsea systemd[1]: Starting OpenBSD Secure Shell server...
Aug 04 18:47:55 node-horsea sshd[901]: Server listening on 0.0.0.0 port 22.
Aug 04 18:47:55 node-horsea sshd[901]: Server listening on :: port 22.
Aug 04 18:47:55 node-horsea systemd[1]: Started OpenBSD Secure Shell server.
Aug 04 18:50:25 node-horsea sshd[1167]: Accepted publickey for ubuntu from 10.172.68.194 port 42610 ssh2: RSA SHA256:7JLFXGJ/lkuhcSu8/OF8mHjZ8wMD1zEfNSuNGM5u2Lk
Aug 04 18:50:25 node-horsea sshd[1167]: pam_unix(sshd:session): session opened for user ubuntu(uid=1000) by (uid=0)

I will mark bug #2004551 as a duplicate of this one. I can also help with the SRU verification, Nick. Just let me know.

Thanks.

[Sergio Durigan Junior (sergiodj)]

FWIW, and just to keep record of the investigation: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/2004551/comments/22

[Łukasz Zemczak (sil2100)]

Okay, sadly this bug is not sufficiently documented. From what I see it changes code in a core part of systemd (core/main.c), and the postinst script, which potentially can have quite the impact. And I see no analysis of regression potential here. What could go wrong? Where should we look for regressions if such were to happen? What other impact could these changes have?

I would like the regression potential section to be added. Also, with how the bug description is formatted right now, it's not entirely clear how the testing of this fix should look like. Is it the #testcase section? What is the expected outcome? I'd like this cleaned up as well.

[Nick Rosbrook (enr0n)]

Sorry, I forgot to update the description before. I have fixed that now.

[Łukasz Zemczak (sil2100)]

Thank you for the updated description, I think this is good enough to get all these into -proposed. A question though: since the postinst change for https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1803391 is moved around, is there a risk of causing the bug to happen again? Should we re-test that bug while verifying the bugfix here?

[Łukasz Zemczak (sil2100)]

Hello Erik, or anyone else affected,

Accepted systemd into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/252.5-2ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Erik, or anyone else affected,

Accepted systemd into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/249.11-0ubuntu3.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (249.11-0ubuntu3.10) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

apt/2.4.10 (armhf)
casync/2+20201210-1build1 (ppc64el)
comitup/1.15-1 (armhf)
dbus/1.12.20-2ubuntu4.1 (armhf)
initramfs-tools/0.140ubuntu13.4 (s390x)
linux-azure-5.19/5.19.0-1027.30~22.04.2 (arm64)
linux-gcp-6.2/6.2.0-1011.11~22.04.3 (arm64)
linux-lowlatency/5.15.0-83.92 (arm64)
linux-lowlatency-hwe-5.19/5.19.0-1030.30 (arm64)
linux-nvidia-tegra/5.15.0-1016.16 (arm64)
linux-oracle-5.19/5.19.0-1027.30 (arm64)
mkosi/unknown (s390x)
munin/2.0.57-1ubuntu2 (armhf)
netplan.io/0.105-0ubuntu2~22.04.3 (arm64)
prometheus-postfix-exporter/unknown (s390x)
samba/2:4.15.13+dfsg-0ubuntu1.4 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (252.5-2ubuntu3.1) for lunar have finished running.
The following regressions have been reported in tests triggered by the package:

apt/2.6.0ubuntu0.1 (armhf)
clutter-1.0/unknown (amd64)
corosync/3.1.7-1ubuntu1 (i386)
cryptsetup/unknown (s390x)
libreswan/unknown (s390x)
libsdl2/unknown (s390x)
linux-lowlatency/6.2.0-1012.12 (arm64)
mutter/44.3-0ubuntu1 (amd64)
netplan.io/0.106-0ubuntu3 (arm64)
samba/2:4.17.7+dfsg-1ubuntu2.2 (arm64)
stunnel4/3:5.68-2 (i386)
stunnel4/unknown (s390x)
systemd-bootchart/234-2 (amd64)
ubuntu-drivers-common/1:0.9.7.1.1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/lunar/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Nick Rosbrook (enr0n)]

I have verified the fix on Lunar using 252.5-2ubuntu3.1 from lunar-proposed:

nr@clean-lunar-amd64:~$ apt-cache policy systemd
systemd:
  Installed: 252.5-2ubuntu3
  Candidate: 252.5-2ubuntu3.1
  Version table:
     252.5-2ubuntu3.1 500
        500 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 Packages
 *** 252.5-2ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu lunar/main amd64 Packages
        100 /var/lib/dpkg/status
nr@clean-lunar-amd64:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-6.2.0-26-generic root=UUID=9122271f-96e1-4551-8279-80165b8e42b8 ro quiet splash --- vt.handoff=7
nr@clean-lunar-amd64:~$ sudo apt install systemd -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libnss-systemd libpam-systemd libsystemd-shared libsystemd0 libudev1 systemd-oomd systemd-resolved systemd-sysv
  systemd-timesyncd udev
Suggested packages:
  systemd-container systemd-homed systemd-userdbd systemd-boot libqrencode4
The following packages will be upgraded:
  libnss-systemd libpam-systemd libsystemd-shared libsystemd0 libudev1 systemd systemd-oomd systemd-resolved
  systemd-sysv systemd-timesyncd udev
11 upgraded, 0 newly installed, 0 to remove and 76 not upgraded.
Need to get 7,789 kB of archives.
After this operation, 7,168 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 systemd-timesyncd amd64 252.5-2ubuntu3.1 [33.9 kB]
Get:2 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 systemd amd64 252.5-2ubuntu3.1 [3,002 kB]
Get:3 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 udev amd64 252.5-2ubuntu3.1 [1,693 kB]
Get:4 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 systemd-sysv amd64 252.5-2ubuntu3.1 [11.5 kB]
Get:5 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 systemd-oomd amd64 252.5-2ubuntu3.1 [38.9 kB]
Get:6 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 libnss-systemd amd64 252.5-2ubuntu3.1 [141 kB]
Get:7 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 libpam-systemd amd64 252.5-2ubuntu3.1 [206 kB]
Get:8 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 systemd-resolved amd64 252.5-2ubuntu3.1 [288 kB]
Get:9 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 libsystemd-shared amd64 252.5-2ubuntu3.1 [1,834 kB]
Get:10 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 libudev1 amd64 252.5-2ubuntu3.1 [151 kB]
Get:11 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 libsystemd0 amd64 252.5-2ubuntu3.1 [390 kB]
[...]
nr@clean-lunar-amd64:~$ apt-cache policy systemd
systemd:
  Installed: 252.5-2ubuntu3.1
  Candidate: 252.5-2ubuntu3.1
  Version table:
 *** 252.5-2ubuntu3.1 500
        500 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     252.5-2ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu lunar/main amd64 Packages
nr@clean-lunar-amd64:~$ grep -Fsx systemd /run/reboot-required.pkgs
systemd
nr@clean-lunar-amd64:~$ reboot
[...]
nr@clean-lunar-amd64:~$ sudo systemctl daemon-reexec
nr@clean-lunar-amd64:~$ journalctl ...

[Nick Rosbrook (enr0n)]

I have verified the fix on Jammy using systemd 249.11-0ubuntu3.10 from jammy-proposed:

nr@clean-jammy-amd64:~$ apt-cache policy systemd
systemd:
  Installed: 249.11-0ubuntu3.9
  Candidate: 249.11-0ubuntu3.10
  Version table:
     249.11-0ubuntu3.10 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
 *** 249.11-0ubuntu3.9 500
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     249.11-0ubuntu3.7 500
        500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     249.11-0ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
nr@clean-jammy-amd64:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.19.0-45-generic root=UUID=32a6b55f-64d3-421b-a77d-2f9a2c6d68be ro quiet splash --- vt.handoff=7
nr@clean-jammy-amd64:~$ sudo apt install systemd -y
[sudo] password for nr:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libflashrom1 libftdi1-2 libllvm13 linux-headers-5.15.0-25
  linux-headers-5.15.0-25-generic linux-image-5.15.0-25-generic
  linux-modules-5.15.0-25-generic linux-modules-extra-5.15.0-25-generic
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libnss-systemd libpam-systemd libsystemd0 systemd-oomd systemd-sysv
  systemd-timesyncd
Suggested packages:
  systemd-container libtss2-rc0
The following packages will be upgraded:
  libnss-systemd libpam-systemd libsystemd0 systemd systemd-oomd systemd-sysv
  systemd-timesyncd
7 upgraded, 0 newly installed, 0 to remove and 217 not upgraded.
Need to get 5,312 kB of archives.
After this operation, 4,096 B disk space will be freed.
Get:1 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 libnss-systemd amd64 249.11-0ubuntu3.10 [133 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 systemd-timesyncd amd64 249.11-0ubuntu3.10 [31.2 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 systemd-sysv amd64 249.11-0ubuntu3.10 [10.5 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 systemd-oomd amd64 249.11-0ubuntu3.10 [34.8 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 libpam-systemd amd64 249.11-0ubuntu3.10 [203 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 systemd amd64 249.11-0ubuntu3.10 [4,581 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 libsystemd0 amd64 249.11-0ubuntu3.10 [319 kB]
[...]
nr@clean-jammy-amd64:~$ apt-cache policy systemd
systemd:
  Installed: 249.11-0ubuntu3.10
  Candidate: 249.11-0ubuntu3.10
  Version table:
 *** 249.11-0ubuntu3.10 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     249.11-0ubuntu3.9 500
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
     249.11-0ubuntu3.7 500
        500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     249.11-0ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd...

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 253.5-1ubuntu4

---------------
systemd (253.5-1ubuntu4) mantic; urgency=medium

  * Revert "Drop Provides: time-daemon for bin:systemd-timesyncd"
    File: debian/control
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=e88d9f0241a307a3c1b4bcaf4bf29799d984aa3c

 -- Nick Rosbrook <email address hidden> Fri, 25 Aug 2023 16:36:26 -0400

[Erik Wasser (ewasser)]

@Łukasz: Thanks for the fix. I've tried the packages systemd (and other packages from the source ball) in version "252.5-2ubuntu3.1".

And the bug went away. Great. Thank you and the other hard-working people.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 252.5-2ubuntu3.1

---------------
systemd (252.5-2ubuntu3.1) lunar; urgency=medium

  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=2a443ddd5b67596647e463eee64d58c3c71bbf08
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=064c7c00aa133b697ab3c0082b825e2f197f91cc
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=45541ccbc16ed30b6a120cda7401f69cce164d54
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ddea3bcbca3421299b13cfe897157cfbf9f0f06d
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=e1a08bb427e19ced6c16f054ebdee43ccbc9c528
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=8235427fd8859ee393a08d425e1a52e1ebcbe2f1
  * sd-bus: bus_message_type_from_string is not pure (LP: #2031683)
    File: debian/patches/lp2031683-sd-bus-bus_message_type_from_string-is-not-pure.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=2d65b6c74864976a2bb42c884baf75d1674215fc

 -- Nick Rosbrook <email address hidden> Fri, 18 Aug 2023 14:15:03 -0400

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 249.11-0ubuntu3.10

---------------
systemd (249.11-0ubuntu3.10) jammy; urgency=medium

  [ Nick Rosbrook ]
  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=85b2ceddff1a6cc1ddbca8a1b7e5381d146e6313
  * pstore: only try to load efi_pstore module (LP: #2023462)
    File: debian/patches/lp1978079-efi-pstore-not-cleared-on-boot.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7600bbfb1e8a399e5aeb1010a20deda3e5a06c89
  * shutdown: get only active md arrays. (LP: #2025563)
    File: debian/patches/lp2025563-shutdown-get-only-active-md-arrays.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=416a9245c8f0efbedcc4395cada23cb09c685ec3
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=26e85b944da9098e66fc0c39f64ee40254c0c278
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=19ba0f20d311642596dc65fa5d6eb96a2d4be280
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=73e774a2fb99c82df6d0edd770bb84ab735ec2f0
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=cf82f08feea456e1c65895b34bffa8c33d421588
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=be484dab06d590b1792a8f016f4292373d0174b7
  * binfmt: fix systemd-binfmt in LXD containers (LP: #1999275)
    Files:
    - debian/patches/lp1999275/binfmt-check-if-binfmt-is-mounted-before-applying-rules.patch
    - debian/patches/lp1999275/binfmt-util-also-check-if-binfmt-is-mounted-in-read-write.patch
    - debian/patches/lp1999275/binfmt-util-split-out-binfmt_mounted.patch
    - debian/patches/lp1999275/unit-check-more-specific-path-to-be-written-by-systemd-bi.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0b9eadf05ee6db0e009dacbc71521480095880b3

  [ Dan Streetman ]
  * Fix machinectl pull-tar and import-tar (LP: #1977630)
    Author: Dan Streetman
    File: debian/patches/lp1977630-fix_machinectl_pull_tar.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+sourc...

[Steve Langasek (vorlon)]

The release of this SRU has been rolled back in jammy because of bug #2035406 which appears to be a regression introduced in the new version of systemd.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 249.11-0ubuntu3.10

---------------
systemd (249.11-0ubuntu3.10) jammy; urgency=medium

  [ Nick Rosbrook ]
  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=85b2ceddff1a6cc1ddbca8a1b7e5381d146e6313
  * pstore: only try to load efi_pstore module (LP: #2023462)
    File: debian/patches/lp1978079-efi-pstore-not-cleared-on-boot.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7600bbfb1e8a399e5aeb1010a20deda3e5a06c89
  * shutdown: get only active md arrays. (LP: #2025563)
    File: debian/patches/lp2025563-shutdown-get-only-active-md-arrays.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=416a9245c8f0efbedcc4395cada23cb09c685ec3
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=26e85b944da9098e66fc0c39f64ee40254c0c278
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=19ba0f20d311642596dc65fa5d6eb96a2d4be280
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=73e774a2fb99c82df6d0edd770bb84ab735ec2f0
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=cf82f08feea456e1c65895b34bffa8c33d421588
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=be484dab06d590b1792a8f016f4292373d0174b7
  * binfmt: fix systemd-binfmt in LXD containers (LP: #1999275)
    Files:
    - debian/patches/lp1999275/binfmt-check-if-binfmt-is-mounted-before-applying-rules.patch
    - debian/patches/lp1999275/binfmt-util-also-check-if-binfmt-is-mounted-in-read-write.patch
    - debian/patches/lp1999275/binfmt-util-split-out-binfmt_mounted.patch
    - debian/patches/lp1999275/unit-check-more-specific-path-to-be-written-by-systemd-bi.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0b9eadf05ee6db0e009dacbc71521480095880b3

  [ Dan Streetman ]
  * Fix machinectl pull-tar and import-tar (LP: #1977630)
    Author: Dan Streetman
    File: debian/patches/lp1977630-fix_machinectl_pull_tar.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+sourc...