Activity log for bug #1896614

Date Who What changed Old value New value Message
2020-09-22 11:44:43 Victor Tapia bug added bug
2020-09-22 11:44:53 Victor Tapia tags sts
2020-09-22 11:44:59 Victor Tapia nominated for series Ubuntu Bionic
2020-09-22 11:44:59 Victor Tapia bug task added systemd (Ubuntu Bionic)
2020-09-22 13:06:23 Victor Tapia bug watch added https://github.com/systemd/systemd/issues/12956
2020-09-22 13:51:05 Dan Streetman bug added subscriber Dan Streetman
2020-09-23 17:31:42 Dan Streetman systemd (Ubuntu Bionic): assignee Victor Tapia (vtapia)
2020-09-23 17:31:44 Dan Streetman systemd (Ubuntu Bionic): importance Undecided Medium
2020-09-23 17:31:47 Dan Streetman systemd (Ubuntu Bionic): status New In Progress
2020-09-23 17:31:50 Dan Streetman systemd (Ubuntu): status New Fix Released
2020-09-23 17:53:41 Dan Streetman description In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind [impact] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. [test case] set up a 1-cpu VM with Bionic, and configure the system with a ssh key so the user can ssh to localhost. Then run: ubuntu@lp1896614-b:~$ while timeout 5 ssh localhost true; do echo 'reloading'; sudo systemctl restart systemd-logind & sudo systemctl daemon-reload; done that should exit the while loop after only a few attempts. At that point, there should be a running job for systemd-logind, and any logins attempted after the bug is reproduced should also hang waiting for the systemd-logind job to complete, e.g.: ubuntu@lp1896614-b:~$ systemctl list-jobs JOB UNIT TYPE STATE 525 systemd-logind.service start running 669 session-6.scope start waiting 664 session-5.scope start waiting 3 jobs listed. [regression potential] any regression would likely involve services that are Type=dbus failing to complete starting. as with any systemd change, regressions could also involve assertion failures in systemd which causes it to exit. [scope] this is needed only for bionic. TBD - needed for xenial? this is fixed upstream with commit a5a8776ae5e4244b7f5acb2a1bfbe6e0b4d8a870 which is including starting in v243, so it is included already in focal and later. [original description] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind
2020-09-23 17:57:04 Dan Streetman bug task added systemd
2020-09-23 17:59:12 Dan Streetman description [impact] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. [test case] set up a 1-cpu VM with Bionic, and configure the system with a ssh key so the user can ssh to localhost. Then run: ubuntu@lp1896614-b:~$ while timeout 5 ssh localhost true; do echo 'reloading'; sudo systemctl restart systemd-logind & sudo systemctl daemon-reload; done that should exit the while loop after only a few attempts. At that point, there should be a running job for systemd-logind, and any logins attempted after the bug is reproduced should also hang waiting for the systemd-logind job to complete, e.g.: ubuntu@lp1896614-b:~$ systemctl list-jobs JOB UNIT TYPE STATE 525 systemd-logind.service start running 669 session-6.scope start waiting 664 session-5.scope start waiting 3 jobs listed. [regression potential] any regression would likely involve services that are Type=dbus failing to complete starting. as with any systemd change, regressions could also involve assertion failures in systemd which causes it to exit. [scope] this is needed only for bionic. TBD - needed for xenial? this is fixed upstream with commit a5a8776ae5e4244b7f5acb2a1bfbe6e0b4d8a870 which is including starting in v243, so it is included already in focal and later. [original description] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind [impact] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. [test case] set up a 1-cpu VM with Bionic, and configure the system with a ssh key so the user can ssh to localhost. Then run: ubuntu@lp1896614-b:~$ while timeout 5 ssh localhost true; do echo 'reloading'; sudo systemctl restart systemd-logind & sudo systemctl daemon-reload; done that should exit the while loop after only a few attempts. At that point, there should be a running job for systemd-logind, and any logins attempted after the bug is reproduced should also hang waiting for the systemd-logind job to complete, e.g.: ubuntu@lp1896614-b:~$ systemctl list-jobs JOB UNIT TYPE STATE 525 systemd-logind.service start running 669 session-6.scope start waiting 664 session-5.scope start waiting 3 jobs listed. [regression potential] any regression would likely involve services that are Type=dbus failing to complete starting. as with any systemd change, regressions could also involve assertion failures in systemd which causes it to exit. [scope] this is needed only for bionic. this is fixed upstream with commit a5a8776ae5e4244b7f5acb2a1bfbe6e0b4d8a870 which is including starting in v243, so it is included already in focal and later. (per upstream bug) this was introduced by upstream commit 75152a4d6aedbfd3ee8b2d5782b9edf27407622a which was included starting in v237, so this bug is not present in xenial or earlier. [original description] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind
2020-09-24 18:13:26 Bug Watch Updater systemd: status Unknown Fix Released
2020-10-01 18:39:25 Dan Streetman description [impact] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. [test case] set up a 1-cpu VM with Bionic, and configure the system with a ssh key so the user can ssh to localhost. Then run: ubuntu@lp1896614-b:~$ while timeout 5 ssh localhost true; do echo 'reloading'; sudo systemctl restart systemd-logind & sudo systemctl daemon-reload; done that should exit the while loop after only a few attempts. At that point, there should be a running job for systemd-logind, and any logins attempted after the bug is reproduced should also hang waiting for the systemd-logind job to complete, e.g.: ubuntu@lp1896614-b:~$ systemctl list-jobs JOB UNIT TYPE STATE 525 systemd-logind.service start running 669 session-6.scope start waiting 664 session-5.scope start waiting 3 jobs listed. [regression potential] any regression would likely involve services that are Type=dbus failing to complete starting. as with any systemd change, regressions could also involve assertion failures in systemd which causes it to exit. [scope] this is needed only for bionic. this is fixed upstream with commit a5a8776ae5e4244b7f5acb2a1bfbe6e0b4d8a870 which is including starting in v243, so it is included already in focal and later. (per upstream bug) this was introduced by upstream commit 75152a4d6aedbfd3ee8b2d5782b9edf27407622a which was included starting in v237, so this bug is not present in xenial or earlier. [original description] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind [impact] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. [test case] set up a 1-cpu VM with Bionic, and configure the system with a ssh key so the user can ssh to localhost. Then run something like: $ while timeout 5 ssh localhost true; do echo 'reloading'; sudo systemctl restart systemd-logind & sudo systemctl daemon-reload; done if that doesn't work try: $ while timeout 5 ssh localhost true; do echo 'reloading'; sudo sh -c 'systemctl restart systemd-logind & systemctl daemon-reload'; done once the reproducer exits the while loop, there should be a running job for systemd-logind, and any logins attempted after the bug is reproduced should also hang waiting for the systemd-logind job to complete, e.g.: ubuntu@lp1896614-b:~$ systemctl list-jobs JOB UNIT TYPE STATE 525 systemd-logind.service start running 669 session-6.scope start waiting 664 session-5.scope start waiting 3 jobs listed. [regression potential] any regression would likely involve services that are Type=dbus failing to complete starting. as with any systemd change, regressions could also involve assertion failures in systemd which causes it to exit. [scope] this is needed only for bionic. this is fixed upstream with commit a5a8776ae5e4244b7f5acb2a1bfbe6e0b4d8a870 which is including starting in v243, so it is included already in focal and later. (per upstream bug) this was introduced by upstream commit 75152a4d6aedbfd3ee8b2d5782b9edf27407622a which was included starting in v237, so this bug is not present in xenial or earlier. [original description] In certain scenarios, such as high load environments or when "systemctl daemon-reload" runs at the same time a dbus service is starting (e.g. systemd-logind), systemd is not able to track properly when the service has started, keeping the job 'running' forever. The issue appears when systemd runs the "AddMatch" dbus method call to track the service's "NameOwnerChange" once it has already ran. A working instance would look like this: https://pastebin.ubuntu.com/p/868J6WBRQx/ A failing instance would be: https://pastebin.ubuntu.com/p/HhJZ4p8dT5/ I've been able to reproduce the issue on Bionic (237-3ubuntu10.42) running: sudo systemctl daemon-reload & sudo systemctl restart systemd-logind
2020-10-26 15:11:26 Łukasz Zemczak systemd (Ubuntu Bionic): status In Progress Fix Committed
2020-10-26 15:11:28 Łukasz Zemczak bug added subscriber Ubuntu Stable Release Updates Team
2020-10-26 15:11:29 Łukasz Zemczak bug added subscriber SRU Verification
2020-10-26 15:11:33 Łukasz Zemczak tags sts sts verification-needed verification-needed-bionic
2020-11-02 16:07:33 Dan Streetman tags sts verification-needed verification-needed-bionic sts verification-done verification-done-bionic
2020-11-03 23:40:10 Chris Halse Rogers removed subscriber Ubuntu Stable Release Updates Team
2020-11-03 23:42:07 Launchpad Janitor systemd (Ubuntu Bionic): status Fix Committed Fix Released