test_systemd_fsck_with_plymouth_failure passes but marked expected failure

Bug #1829450 reported by Dan Streetman on 2019-05-17
4
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Medium
Dan Streetman
Xenial
Undecided
Unassigned
Bionic
Medium
Unassigned
Cosmic
Medium
Unassigned
Disco
Medium
Unassigned
Eoan
Medium
Dan Streetman

Bug Description

[impact]

this test case was marked as expected failure in the past to try to ignore failures, but it now passes and so causes autopkgtest failures.

[test case]

run autopkgtest and check for the 'test_systemd_fsck_with_plymouth_failure' test, e.g.:

test_systemd_fsck_with_plymouth_failure (__main__.FsckdTest)
Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit ... bash: line 1: 786 Killed /tmp/autopkgtest.84SCbj/build.1op/src/debian/tests/systemd-fsckd 2> >(tee -a /tmp/autopkgtest.84SCbj/systemd-fsckd-stderr >&2) > >(tee -a /tmp/autopkgtest.84SCbj/systemd-fsckd-stdout)
autopkgtest [16:47:57]: test process requested reboot with marker test_systemd_fsck_with_plymouth_failure:0
Connection to 10.44.40.6 closed by remote host.
autopkgtest-virt-ssh: WARNING: ssh connection failed. Retrying in 3 seconds...
test_systemd_fsck_with_plymouth_failure (__main__.FsckdTest)
Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit ... unexpected success

----------------------------------------------------------------------
Ran 1 test in 17.952s

FAILED (unexpected successes=1)

[regression potential]

low; test case expectation fix only.

Dan Streetman (ddstreet) on 2019-05-17
Changed in systemd (Ubuntu Bionic):
status: New → In Progress
Changed in systemd (Ubuntu Cosmic):
status: New → In Progress
Changed in systemd (Ubuntu Disco):
status: New → In Progress
Changed in systemd (Ubuntu Eoan):
status: New → In Progress
Changed in systemd (Ubuntu Bionic):
importance: Undecided → Medium
Changed in systemd (Ubuntu Cosmic):
importance: Undecided → Medium
Changed in systemd (Ubuntu Disco):
importance: Undecided → Medium
Changed in systemd (Ubuntu Eoan):
importance: Undecided → Medium
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Disco):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Cosmic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Xenial):
status: New → Invalid
Dimitri John Ledkov (xnox) wrote :

"and so causes autopkgtest failures." is not true.

From http://autopkgtest.ubuntu.com/packages/systemd/bionic/amd64
E.g. https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/s/systemd/20190529_090547_1f40a@/log.gz

Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit ... unexpected success

FAILED (unexpected successes=1)

autopkgtest [09:05:38]: test systemd-fsckd: -----------------------]
systemd-fsckd PASS

Note systemd-fsckd only fails on failures & errors, and passed with unexpected successes:
if len(result.failures) != 0 or len(result.errors) != 0:
    return_code = 1

I also wonder what has changed to make this test case reliable, in the past it used to be flakey, that's why i switched it to expectedfailure. I guess it's ok to reenable this in devel.

I see no need to reenable it in stable series. It has a regression potential, since it used to be flakey, it may start to be flakey again blocking SRU migrations.

Changed in systemd (Ubuntu Eoan):
status: In Progress → Fix Committed
Dan Streetman (ddstreet) wrote :

> I see no need to reenable it in stable series. It has a regression potential,
> since it used to be flakey, it may start to be flakey again blocking SRU migrations.

er, isn't that the point of these tests, if they start failing again we should investigate and fix the problem, instead of ignoring them? Maybe I'm not understanding what you mean, and if so, sorry.

Dan Streetman (ddstreet) on 2019-05-29
Changed in systemd (Ubuntu Disco):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Cosmic):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Bionic):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Disco):
status: In Progress → Won't Fix
Changed in systemd (Ubuntu Cosmic):
status: In Progress → Triaged
Changed in systemd (Ubuntu Bionic):
status: In Progress → Won't Fix
Changed in systemd (Ubuntu Cosmic):
status: Triaged → Won't Fix
Launchpad Janitor (janitor) wrote :
Download full text (3.5 KiB)

This bug was fixed in the package systemd - 240-6ubuntu9

---------------
systemd (240-6ubuntu9) eoan; urgency=medium

  * Fix typpo in storage test.
    File: debian/tests/storage
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=f28aa5fe4ab175b99b6ea702559c59ca473b4ca8

  * Fix bashism
    File: debian/extra/dhclient-enter-resolved-hook
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0725c1169ddde4f41cacba7af3e546704e2206be

systemd (240-6ubuntu8) eoan; urgency=medium

  * Only restart resolved on changes in dhclient enter hook.
    This prevents spurious restarts of resolved on rebounds when
    the addresses did not change. (LP: #1805183)
    Author: Julian Andres Klode
    File: debian/extra/dhclient-enter-resolved-hook
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=258893bae8cbb12670e4807636fe8f7e9fb5407a

  * Wait for cryptsetup unit to start, before stopping.
    Patch from cascardo. Plus small refactor for readability. (LP: #1814373)
    File: debian/tests/storage
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b65aa350be7e61c65927fbc0921a750fcfaa51cd

  * Wait for systemctl is-system-running state.
    File: debian/tests/boot-smoke
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=776998f1f55c445b6e385cab69a4219c42d00838

systemd (240-6ubuntu7) eoan; urgency=medium

  * Revert "Add check to switch VTs only between K_XLATE or K_UNICODE"
    This reverts commit 60407728a1a453104e3975ecfdf25a254dd7cc44.
    Files:
    - debian/patches/Add-check-to-switch-VTs-only-between-K_XLATE-or-K_UNICODE.patch
    - debian/patches/Move-verify_vc_kbmode-to-terminal-util.c-as-vt_verify_kbm.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=18029ab5ff436bfb3b401f24cd1e3a4cf2a1579c

  * Cherrypick missing systemd-stable patches to unbreak wireguard peer endpoints.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1825378)
    Author: Dan Streetman
    Files:
    - debian/patches/network-wireguard-fixes-sending-wireguard-peer-setti.patch
    - debian/patches/network-wireguard-use-sd_netlink_message_append_sock.patch
    - debian/patches/sd-netlink-introduce-sd_netlink_message_append_socka.patch
    - debian/patches/test-network-add-more-checks-in-NetworkdNetDevTests..patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=4046f515e40c4dc80d18d2303466737f1f451f11

  * Remove expected failure from passing test.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1829450)
    Author: Dan Streetman
    File: debian/tests/systemd-fsckd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=c43b12037d08555dc1d26593307726d7c7992df0

  * Fix false negative checking for running jobs after boot.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1825997)
    Author: Dan Streetman
    File: debian/tests/boot-smoke
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=aeb01631efbaf3fe851dee15d496e0b66b5c347f

  * Cherrypick ask-password: prevent buffer ...

Read more...

Changed in systemd (Ubuntu Eoan):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers