| 2019-04-23 14:30:03 |
Dan Streetman |
bug |
|
|
added bug |
| 2019-04-23 14:55:04 |
Dan Streetman |
nominated for series |
|
Ubuntu Disco |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
bug task added |
|
systemd (Ubuntu Disco) |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
bug task added |
|
systemd (Ubuntu Bionic) |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
nominated for series |
|
Ubuntu Eoan |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
bug task added |
|
systemd (Ubuntu Eoan) |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
nominated for series |
|
Ubuntu Cosmic |
|
| 2019-04-23 14:55:04 |
Dan Streetman |
bug task added |
|
systemd (Ubuntu Cosmic) |
|
| 2019-04-23 14:55:13 |
Dan Streetman |
systemd (Ubuntu Eoan): status |
New |
In Progress |
|
| 2019-04-23 14:55:15 |
Dan Streetman |
systemd (Ubuntu Disco): status |
New |
In Progress |
|
| 2019-04-23 14:55:17 |
Dan Streetman |
systemd (Ubuntu Cosmic): status |
New |
In Progress |
|
| 2019-04-23 14:55:19 |
Dan Streetman |
systemd (Ubuntu Bionic): status |
New |
In Progress |
|
| 2019-04-23 14:55:20 |
Dan Streetman |
systemd (Ubuntu Eoan): importance |
Undecided |
Low |
|
| 2019-04-23 14:55:21 |
Dan Streetman |
systemd (Ubuntu Disco): importance |
Undecided |
Low |
|
| 2019-04-23 14:55:23 |
Dan Streetman |
systemd (Ubuntu Cosmic): importance |
Undecided |
Low |
|
| 2019-04-23 14:55:24 |
Dan Streetman |
systemd (Ubuntu Bionic): importance |
Undecided |
Low |
|
| 2019-04-23 14:55:26 |
Dan Streetman |
systemd (Ubuntu Eoan): assignee |
|
Dan Streetman (ddstreet) |
|
| 2019-04-23 14:55:28 |
Dan Streetman |
systemd (Ubuntu Disco): assignee |
|
Dan Streetman (ddstreet) |
|
| 2019-04-23 14:55:30 |
Dan Streetman |
systemd (Ubuntu Cosmic): assignee |
|
Dan Streetman (ddstreet) |
|
| 2019-04-23 14:55:32 |
Dan Streetman |
systemd (Ubuntu Bionic): assignee |
|
Dan Streetman (ddstreet) |
|
| 2019-04-23 15:10:21 |
Dan Streetman |
summary |
boot-smoke test timeout too short for overloaded autopkgtest.ubuntu.com |
boot-smoke running jobs test based on bad assumption |
|
| 2019-04-23 15:10:52 |
Dan Streetman |
summary |
boot-smoke running jobs test based on bad assumption |
boot-smoke fails due to running jobs |
|
| 2019-04-23 15:22:31 |
Dan Streetman |
description |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, but only gives 35 seconds for each boot. On loaded systems this is too short.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
longer autopkgtest times.
[other info]
i can't reproduce this failure locally, but it seems to happen intermittently on the adt setup. Therefore, I don't know for sure that the short timeout is actually the cause of the problem, but it certainly seems likely - 35 seconds really isn't very long for a full reboot and for systemd to finish starting all services, especially on the highly loaded autopkgtest.ubuntu.com systems.
There should be no harm, other than delaying an actual failure, from extending the timeout. The test case checks each second if all services have finished starting, so on success case it won't wait any longer than it currently does. |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
While increasing the timeout isn't guaranteed to stop the boot-smoke failures due to still-running jobs, the logs show it certainly should help.
If we continue to get failures for still-running jobs, it probably should just be made a non-failing check. |
|
| 2019-04-23 15:29:46 |
Dan Streetman |
description |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
While increasing the timeout isn't guaranteed to stop the boot-smoke failures due to still-running jobs, the logs show it certainly should help.
If we continue to get failures for still-running jobs, it probably should just be made a non-failing check. |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
The timeout waiting for is-system-running is actually probably fine; what is needed is another timeout while checking list-jobs, after we know that the system is running. Another timeout should let any new jobs started after we reached running complete. |
|
| 2019-04-23 15:40:26 |
Dan Streetman |
description |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
The timeout waiting for is-system-running is actually probably fine; what is needed is another timeout while checking list-jobs, after we know that the system is running. Another timeout should let any new jobs started after we reached running complete. |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
Another wait is needed when checking for remaining running jobs. Or, the running jobs check could be removed entirely, and we can just trust that systemd correctly knows when it has reached running|degraded state. |
|
| 2019-04-23 16:08:40 |
Dan Streetman |
tags |
|
ddstreet-next |
|
| 2019-05-02 19:18:13 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~ddstreet/ubuntu/+source/systemd/+git/systemd/+merge/366857 |
|
| 2019-05-02 19:22:45 |
Dan Streetman |
systemd (Ubuntu Eoan): assignee |
Dan Streetman (ddstreet) |
|
|
| 2019-05-14 18:27:05 |
Dan Streetman |
systemd (Ubuntu Eoan): assignee |
|
Dan Streetman (ddstreet) |
|
| 2019-05-17 20:21:18 |
Dan Streetman |
nominated for series |
|
Ubuntu Xenial |
|
| 2019-05-17 20:21:18 |
Dan Streetman |
bug task added |
|
systemd (Ubuntu Xenial) |
|
| 2019-05-17 20:21:24 |
Dan Streetman |
systemd (Ubuntu Xenial): status |
New |
Invalid |
|
| 2019-05-29 14:30:24 |
Dan Streetman |
description |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
Another wait is needed when checking for remaining running jobs. Or, the running jobs check could be removed entirely, and we can just trust that systemd correctly knows when it has reached running|degraded state. |
[impact]
boot-smoke test reboots 5 times and verifies systemd is fully started up after each boot, including checking if there are any running jobs (with list-jobs). However, this test makes the assumption that no further jobs will be started after systemd reaches 'running' (or 'degraded') state, which is a false assumption.
[test case]
see various boot-smoke failures in autopkgtest.ubuntu.com
[regression potential]
possible false-positive or false-negative autopkgtest results.
[other info]
Note: This patch is not required for debian, because debian's boot-smoke does not include the wait for systemctl is-system-running.
The problem appears to be that systemd reaches 'running' (or 'degraded') state, and then other systemd services are started. This confuses the boot-smoke test, because it sees that 'is-system-running' is done, but then it sees running jobs, which fails the test.
What is starting jobs after systemd reaches running state appears to be X inside the test system. There are various services started by gnome-session and dbus-daemon. Additionally, from the artifacts of one example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/s/systemd/20190416_171327_478f6@/artifacts.tar.gz
the artifacts/journal.txt shows that after the boot-smoke test causes the reboot and then re-ssh into the system after the reboot, it only gives the test system 9 seconds before deciding it has failed, and only 4 seconds after ssh'ing into the rebooted test system.
Another wait is needed when checking for remaining running jobs. Or, the running jobs check could be removed entirely, and we can just trust that systemd correctly knows when it has reached running|degraded state. |
|
| 2019-05-29 18:06:33 |
Dimitri John Ledkov |
systemd (Ubuntu Eoan): status |
In Progress |
Fix Committed |
|
| 2019-05-30 12:29:54 |
Dan Streetman |
tags |
ddstreet-next |
|
|
| 2019-05-31 13:33:18 |
Timo Aaltonen |
systemd (Ubuntu Disco): status |
In Progress |
Fix Committed |
|
| 2019-05-31 13:33:20 |
Timo Aaltonen |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2019-05-31 13:33:23 |
Timo Aaltonen |
bug |
|
|
added subscriber SRU Verification |
| 2019-05-31 13:33:26 |
Timo Aaltonen |
tags |
|
verification-needed verification-needed-disco |
|
| 2019-05-31 13:35:34 |
Timo Aaltonen |
systemd (Ubuntu Cosmic): status |
In Progress |
Fix Committed |
|
| 2019-05-31 13:35:40 |
Timo Aaltonen |
tags |
verification-needed verification-needed-disco |
verification-needed verification-needed-cosmic verification-needed-disco |
|
| 2019-05-31 13:42:10 |
Timo Aaltonen |
systemd (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2019-05-31 13:42:15 |
Timo Aaltonen |
tags |
verification-needed verification-needed-cosmic verification-needed-disco |
verification-needed verification-needed-bionic verification-needed-cosmic verification-needed-disco |
|
| 2019-06-03 14:11:45 |
Dan Streetman |
tags |
verification-needed verification-needed-bionic verification-needed-cosmic verification-needed-disco |
verification-done verification-done-bionic verification-done-cosmic verification-done-disco |
|
| 2019-06-05 01:33:15 |
Launchpad Janitor |
systemd (Ubuntu Eoan): status |
Fix Committed |
Fix Released |
|
| 2019-06-10 14:21:27 |
Launchpad Janitor |
systemd (Ubuntu Disco): status |
Fix Committed |
Fix Released |
|
| 2019-06-10 14:21:41 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2019-06-10 14:43:41 |
Launchpad Janitor |
systemd (Ubuntu Cosmic): status |
Fix Committed |
Fix Released |
|
| 2019-06-10 14:49:24 |
Launchpad Janitor |
systemd (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
