systemd segv coredump, reboot/poweroff ignored.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| systemd |
Fix Released
|
Unknown
|
||
| systemd (Ubuntu) |
Undecided
|
Unassigned | ||
| Bionic |
Medium
|
Dan Streetman | ||
| Eoan |
Medium
|
Dan Streetman | ||
| Focal |
Medium
|
Dan Streetman | ||
| Groovy |
Undecided
|
Unassigned |
Bug Description
[impact]
during systemctl daemon-reload, systemd sometimes segfaults while serializing a service.
[test case]
problem is intermittent, but the upstream bug suggests a reproducer. Also see original description.
[regression potential]
this adjusts serialization, so any regression would likely occur during serialization/
[scope]
This is needed for Bionic and later.
this was introduced by commit e266c068b5597e1
The PR to fix this is https:/
[other info]
there is an additional PR that further avoids the error, but has not yet been accepted, and may not be accepted as it may not be necessary.
https:/
[original description]
Proximate syslog:
Apr 18 16:26:56 gate2 rssmonitor.
Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
un/dbus/
Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
se update the unit file accordingly.
Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
id; please update the unit file accordingly.
Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
un/fail2ban/
Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
tmonger.pid; please update the unit file accordingly.
Apr 18 16:26:56 gate2 systemd[1]: Reloading.
Apr 18 16:26:56 gate2 kernel: [ 454.966336] systemd[1]: segfault at 50 ip 000055946ed0d6f0 sp 00007fff8409d210 error 4 in systemd[
Apr 18 16:26:56 gate2 kernel: [ 454.966354] Code: a8 48 8b 75 a0 c7 45 ac 00 00 00 00 48 8b 94 c7 a0 04 00 00 48 89 45 88 48 89 f0 48 39 d6 74 17 66 2e 0f 1f 84 00 00 00 00 00 <48
> 8b 40 50 83 45 ac 01 48 39 c2 75 f3 48 8b 45 a0 31 db 4c 8d 7d
Apr 18 16:26:56 gate2 rssmonitor.
Apr 18 16:26:57 gate2 rssmonitor.
Apr 18 16:27:23 gate2 sssd[nss]: Enumeration requested but not enabled
Apr 18 16:27:24 gate2 kernel: [ 482.503042] printk: systemd: 43 output lines suppressed due to ratelimiting
Apr 18 16:27:24 gate2 systemd[1]: Caught <SEGV>, dumped core as pid 2034.
ProblemType: Bug
DistroRelease: Ubuntu 19.10
Package: systemd 242-7ubuntu3.7
ProcVersionSign
Uname: Linux 5.3.0-46-generic x86_64
ApportVersion: 2.20.11-0ubuntu8.8
Architecture: amd64
Date: Sat Apr 18 18:09:06 2020
InstallationDate: Installed on 2019-11-01 (169 days ago)
InstallationMedia: Ubuntu-MATE 19.10 "Eoan Ermine" - Release amd64 (20191017)
Lsusb:
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
ProcEnviron:
SHELL=/bin/bash
LANG=en_US.UTF-8
TERM=xterm-
XDG_RUNTIME_
PATH=(custom, no user)
ProcKernelCmdLine: BOOT_IMAGE=
SourcePackage: systemd
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.12.0-1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.
dmi.modalias: dmi:bvnSeaBIOS:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.
dmi.sys.vendor: QEMU
mtime.conffile.
Harry Coin (hcoin) wrote : | #1 |
Harry Coin (hcoin) wrote : | #2 |
Harry Coin (hcoin) wrote : | #3 |
This has been happening perhaps 3 times every 7 days, no obvious cause I can see. The system is a virtual machine that's just a router. No user activity, nobody logged in to the mate gui. Wish I had more to offer by way of a clue but that's it.
Dan Streetman (ddstreet) wrote : | #4 |
Are you able to manually trigger the segfault by running 'sudo systemctl daemon-reload' ?
Harry Coin (hcoin) wrote : | #5 |
Just tried it three times, all normal, nothing in the logs except a few netfilter reports of correctly dropped packets.
Launchpad Janitor (janitor) wrote : | #6 |
Status changed to 'Confirmed' because the bug affects multiple users.
Changed in systemd (Ubuntu): | |
status: | New → Confirmed |
dinamic (dinamic6661) wrote : | #7 |
same here but on a PC
kernel: [58217.847336] systemctl[2787]: segfault at 0 ip 0000000000000000 sp 00007ffc4b4f9df8 error 14 in systemctl[
crash log
Harry Coin (hcoin) wrote : | #8 |
And again:
root@noc1:~#
Broadcast message from <email address hidden> (Tue 2020-04-28 12:32:15 CDT):
systemd[1]: Caught <SEGV>, dumped core as pid 25084.
Broadcast message from <email address hidden> (Tue 2020-04-28 12:32:15 CDT):
systemd[1]: Freezing execution.
root@noc1:~#
from the syslog:
Apr 28 12:31:46 noc1 rssmonitor.
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: /lib/systemd/
Apr 28 12:31:46 noc1 systemd[1]: Reloading.
Apr 28 12:31:46 noc1 kernel: [37890.433153] systemd[1]: segfault at 50 ip 000055c2dd32d6f0 sp 00007ffef173c7b0 error 4 in systemd[
Apr 28 12:31:46 noc1 kernel: [37890.433172] Code: a8 48 8b 75 a0 c7 45 ac 00 00 00 00 48 8b 94 c7 a0 04 00 00 48 89 45 88 48 89 f0 48 39 d6 74 17 66 2e 0f 1f 84 00 00 00 00 00 <48> 8b 40 50 83 45 ac 01 48 39 c2 75 f3 48 8b 45 a0 31 db 4c 8d 7d
Apr 28 12:31:47 noc1 rssmonitor.
The last entry means the systemctl daemon-reload command returned a normal 0 exit code.
Changed in systemd (Ubuntu): | |
status: | Confirmed → Fix Released |
Changed in systemd (Ubuntu Eoan): | |
assignee: | nobody → Dan Streetman (ddstreet) |
importance: | Undecided → Medium |
status: | New → In Progress |
description: | updated |
Changed in systemd: | |
status: | Unknown → Fix Released |
Changed in systemd (Ubuntu Focal): | |
status: | New → In Progress |
Changed in systemd (Ubuntu Bionic): | |
status: | New → In Progress |
assignee: | nobody → Dan Streetman (ddstreet) |
Changed in systemd (Ubuntu Focal): | |
assignee: | nobody → Dan Streetman (ddstreet) |
importance: | Undecided → Medium |
Changed in systemd (Ubuntu Bionic): | |
importance: | Undecided → Medium |
Changed in systemd (Ubuntu Groovy): | |
status: | Fix Released → New |
Balint Reczey (rbalint) wrote : | #9 |
Fixed in ubuntu/
Changed in systemd (Ubuntu Groovy): | |
status: | New → Fix Released |
Hello Harry, or anyone else affected,
Accepted systemd into focal-proposed. The package will build now and be available at https:/
Please help us by testing this new package. See https:/
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-
Further information regarding the verification process can be found at https:/
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.
Changed in systemd (Ubuntu Focal): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed verification-needed-focal |
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/245.4-4ubuntu3.1) | #11 |
All autopkgtests for the newly accepted systemd (245.4-4ubuntu3.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:
hddemux/unknown (armhf)
pyudev/unknown (armhf)
python-
prometheus/
gvfs/unknown (armhf)
indicator-
gpsd/unknown (armhf)
samba/unknown (armhf)
python-
golang-
pystemd/unknown (armhf)
umockdev/0.14.1-1 (armhf)
remctl/unknown (armhf)
icinga2/unknown (armhf)
gnome-desktop3/
munin/2.
haproxy/unknown (armhf)
python-
gvfs/1.
Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUp
https:/
[1] https:/
Thank you!
Dan Streetman (ddstreet) wrote : | #12 |
@hcoin are you still on Eoan or have you moved up to Focal yet? If you're on Focal, could you test the version from -proposed?
Harry Coin (hcoin) wrote : Re: [Bug 1873607] Re: systemd segv coredump, reboot/poweroff ignored. | #13 |
Still on eoan. Staying with Ubuntu depends on freeipa support.
Presently only the freeipa client is supported past eoan, so the future
is unclear. Can the fix be backported?
On 5/13/20 3:03 PM, Dan Streetman wrote:
> @hcoin are you still on Eoan or have you moved up to Focal yet? If
> you're on Focal, could you test the version from -proposed?
>
Dan Streetman (ddstreet) wrote : | #14 |
> Still on eoan. Staying with Ubuntu depends on freeipa support.
> Presently only the freeipa client is supported past eoan, so the future
> is unclear.
oh my, i wasn't aware of bug 1875114. that does not seem good at all.
> Can the fix be backported?
yep, it's in the upload queue for eoan, so should make it into eoan-proposed sometime soon.
Hello Harry, or anyone else affected,
Accepted systemd into eoan-proposed. The package will build now and be available at https:/
Please help us by testing this new package. See https:/
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-
Further information regarding the verification process can be found at https:/
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.
Changed in systemd (Ubuntu Eoan): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed-eoan |
Brian Murray (brian-murray) wrote : | #16 |
Hello Harry, or anyone else affected,
Accepted systemd into bionic-proposed. The package will build now and be available at https:/
Please help us by testing this new package. See https:/
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-
Further information regarding the verification process can be found at https:/
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.
Changed in systemd (Ubuntu Bionic): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed-bionic |
Harry Coin (hcoin) wrote : Re: [Bug 1873607] Re: systemd segv coredump, reboot/poweroff ignored. | #17 |
Trying it now on 1 of 4 previously affected systems. Accepted all
'proposed' items, including changes to libvirt. Will advise.
On 5/14/20 3:19 PM, Brian Murray wrote:
> Hello Harry, or anyone else affected,
>
> Accepted systemd into eoan-proposed. The package will build now and be
> available at https:/
> in a few hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https:/
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested, what testing has been
> performed on the package and change the tag from verification-
> eoan to verification-
> please add a comment stating that, and change the tag to verification-
> failed-eoan. In either case, without details of your testing we will not
> be able to proceed.
>
> Further information regarding the verification process can be found at
> https:/
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> ** Changed in: systemd (Ubuntu Eoan)
> Status: In Progress => Fix Committed
>
> ** Tags added: verification-
>
All autopkgtests for the newly accepted systemd (242-7ubuntu3.9) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:
nghttp2/unknown (armhf)
pulseaudio/unknown (armhf)
docker.io/unknown (armhf)
csync2/unknown (armhf)
network-
dovecot/unknown (armhf)
open-iscsi/unknown (armhf)
netplan.io/unknown (armhf)
asterisk/
tinyssh/unknown (armhf)
puppet/unknown (armhf)
corosync/unknown (armhf)
python-
nextepc/unknown (armhf)
debci/unknown (armhf)
pyudev/unknown (armhf)
cups/unknown (armhf)
umockdev/0.13.2-1 (armhf)
cockpit/unknown (ppc64el)
network-
nftables/unknown (armhf)
prometheus/unknown (armhf)
corosync-
prometheus-
python-
prometheus-
ndctl/unknown (armhf)
prometheus-
dpdk/unknown (armhf)
nextepc/
dbus/unknown (armhf)
openssh/
openrazer/unknown (armhf)
gvfs/1.
snapd/2.42.1+19.10 (i386)
prometheus-
prometheus-
python-
systemd/
Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUp
https:/
[1] https:/
Thank you!
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.41) | #19 |
All autopkgtests for the newly accepted systemd (237-3ubuntu10.41) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:
python-
policykit-1/unknown (armhf)
multipath-
debci/unknown (ppc64el)
netplan.
pdns-recursor/
umockdev/0.11.1-1 (armhf)
sssd/unknown (armhf)
linux-raspi2-
suricata/unknown (armhf)
lxc/unknown (armhf)
casync/
openssh/
python-
puppet/unknown (armhf)
prometheus-
lxc/3.0.
postgresql-
polkit-qt-1/unknown (armhf)
munin/unknown (armhf)
systemd/
pulseaudio/unknown (armhf)
php7.2/unknown (armhf)
Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUp
https:/
[1] https:/
Thank you!
Dan Streetman (ddstreet) wrote : | #20 |
@hcoin I assume no more segfaults with the updated pkg? can we mark this verified for eoan?
Harry Coin (hcoin) wrote : Re: [Bug 1873607] Re: systemd segv coredump, reboot/poweroff ignored. | #21 |
No issues so far. Thanks for checking.
On May 20, 2020 5:30:40 PM CDT, Dan Streetman <email address hidden> wrote:
>@hcoin I assume no more segfaults with the updated pkg? can we mark
>this verified for eoan?
>
>--
>You received this bug notification because you are subscribed to the
>bug
>report.
>https:/
>
>Title:
> systemd segv coredump, reboot/poweroff ignored.
>
>Status in systemd:
> Fix Released
>Status in systemd package in Ubuntu:
> Fix Released
>Status in systemd source package in Bionic:
> Fix Committed
>Status in systemd source package in Eoan:
> Fix Committed
>Status in systemd source package in Focal:
> Fix Committed
>Status in systemd source package in Groovy:
> Fix Released
>
>Bug description:
> [impact]
>
> during systemctl daemon-reload, systemd sometimes segfaults while
> serializing a service.
>
> [test case]
>
> problem is intermittent, but the upstream bug suggests a reproducer.
> Also see original description.
>
> [regression potential]
>
> this adjusts serialization, so any regression would likely occur
> during serialization/
>
> [scope]
>
> This is needed for Bionic and later.
>
> this was introduced by commit e266c068b5597e1
> which is included starting in v234. The commit to fix it is
> e9da62b18af647b
> any release.
>
> The PR to fix this is https:/
>
> [other info]
>
>there is an additional PR that further avoids the error, but has not
>yet been accepted, and may not be accepted as it may not be necessary.
> https:/
>
> [original description]
>
> Proximate syslog:
>
>Apr 18 16:26:56 gate2 rssmonitor.
>netns del
>Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
>ListenStream= references a path below legacy directory /var/run/,
>updating /var/run/
> un/dbus/
>Apr 18 16:26:56 gate2 systemd[1]: /lib/systemd/
>PIDFile= references a path below legacy directory /var/run/, updating
>/var/run/sssd.pid → /run/sssd.pid; plea
> se update the unit file accordingly.
>Apr 18 16:26:56 gate2 systemd[1]:
>/lib/systemd/
>legacy directory /var/run/, updating /var/run/
>/run/oddjobd.p
> id; please update the unit file accordingly.
>Apr 18 16:26:56 gate2 systemd[1]:
>/lib/systemd/
>below legacy directory /var/run/, updating
>/var/run/
> un/fail2ban/
>Apr 18 16:26:56 gate2 systemd[1]:
>/lib/systemd/
>below legacy directory /var/run/, updating /var/run/
>/run/cer
> tmonger.pid; please update the unit file accordingly.
> Apr 18 16:26:56 gate2 systemd[1]: Reloading.
>Apr 18 16:26:56 gate2 kernel: [ 454.966336] systemd[1]: segfault at 50
>ip ...
tags: |
added: verification-done-eoan removed: verification-needed-eoan |
Dan Streetman (ddstreet) wrote : | #22 |
I looped daemon-reexec and daemon-reload for a while on both bionic and focal and saw no segfaults with the updated packages. As @hcoin was able to reproduce this on Eoan and the updated pkg appears to have fixed it there, and I was unable to reproduce any segfaults or other issues, I'm marking as verified for b and f also.
tags: |
added: verification-done verification-done-bionic verification-done-focal removed: verification-needed verification-needed-bionic verification-needed-focal |
The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.
Launchpad Janitor (janitor) wrote : | #24 |
This bug was fixed in the package systemd - 245.4-4ubuntu3.1
---------------
systemd (245.4-4ubuntu3.1) focal; urgency=medium
* d/p/lp1867375/
d/p/
d/p/
d/p/
d/p/
d/p/
- Add UseGateway= parameter and default to value of UseRoutes, to restore
backwards compatibility with old UseRoutes= behavior (LP: #1867375)
* d/p/lp1860926-
- default ignore_carrier_loss to value of configure_
so carrier drop during configuration doesn't break networking
(LP: #1860926)
* d/e/initramfs-
- Follow symlinks when finding link files to copy into initramfs
(LP: #1868892)
* d/p/lp1873607/
d/
- Avoid segfault during serialization (LP: #1873607)
-- Dan Streetman <email address hidden> Thu, 07 May 2020 09:21:22 -0400
Changed in systemd (Ubuntu Focal): | |
status: | Fix Committed → Fix Released |
Launchpad Janitor (janitor) wrote : | #25 |
This bug was fixed in the package systemd - 242-7ubuntu3.9
---------------
systemd (242-7ubuntu3.9) eoan; urgency=medium
* d/p/lp1867375/
d/p/
d/p/
d/p/
- Move gateway ignoring from UseRoutes= to UseGateway= (LP: #1867375)
* d/p/lp1873607/
d/
- Avoid segfault during serialization (LP: #1873607)
* d/p/lp1877271-
- enable ipv6 for interface when needed (LP: #1877271)
* d/p/lp1860926-
- IgnoreCarrierLoss default to ConfigureWithou
-- Dan Streetman <email address hidden> Fri, 08 May 2020 12:28:53 -0400
Changed in systemd (Ubuntu Eoan): | |
status: | Fix Committed → Fix Released |
Launchpad Janitor (janitor) wrote : | #26 |
This bug was fixed in the package systemd - 237-3ubuntu10.41
---------------
systemd (237-3ubuntu10.41) bionic; urgency=medium
[ Dan Streetman ]
* d/p/lp1867375/
d/p/
d/p/
- Move gateway ignoring from UseRoutes= to UseGateway= (LP: #1867375)
* d/p/lp1873607/
- Avoid segfault during serialization (LP: #1873607)
* d/p/lp1529152/
d/
d/
d/
d/
- fix slow systemctl tab completion (LP: #1529152)
* d/p/lp1877159-
- avoid kernel err msg setting wireguard param (LP: #1877159)
[ Heitor Alves de Siqueira ]
* d/p/lp1876600-
- deal with dbus cookie overruns (LP: #1876600)
-- Heitor Alves de Siqueira <email address hidden> Sun, 03 May 2020 11:30:25 +0000
Changed in systemd (Ubuntu Bionic): | |
status: | Fix Committed → Fix Released |
crash log