storage autopkgtest always fails with linux-{kvm,azure,etc} kernels that don't include scsi_debug module

[Dan Streetman]

Note: this bug touches on multiple packages:
- autopkgtest-cloud: merge request open to install 'modules-extra' package
- systemd: update autopkgtest to skip test if scsi_debug module isn't available (SRU template below is for this change)
- linux-kvm: has CONFIG_SCSI_DEBUG unset, but would be good to have scsi_debug module built for this kernel

[SRU template for systemd]

[impact]

the 'storage' autopkgtest requires the 'scsi_debug' module, and if it can't load it (and it's not built-in), the test fails. On some custom kernels, this module is included in the new 'linux-modules-extra-*' deb which isn't installed by default, so the test always fails when doing a reverse-dep autopkgtest using such a kernel's meta pkg.

Additionally, some custom kernels, like linux-kvm, have SCSI_DEBUG disabled entirely; for those kernels, it will never be possible to load the scsi_debug module and the test should be skipped.

[test case]

run the 'storage' autopkgtest with a custom kernel that doesn't provide the scsi_debug module in the standard linux-modules deb, or check the autopkgtest.u.c history, for example:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/amd64/s/systemd/20191209_150835_237a5@/log.gz

[regression potential]

as this modifies one of the autopkgtest --setup-commands, a regression would most likely cause immediate test failure due to failing to install the proper linux-* debs.

For the systemd 'storage' test change, this only skips the test if the scsi_debug module can't be loaded (and isn't already present, i.e. loaded or built-in). Any regression potential from that is low, but likely would involve incorrect test skips and/or missing valid problems due to skipping the test.

[scope]

systemd in all releases needs to have its 'storage' test updated

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 244.1-0ubuntu2

---------------
systemd (244.1-0ubuntu2) focal; urgency=medium

  [ Dimitri John Ledkov ]
  * shutdown: do not detach autoclear loopback devices
    Author: Dimitri John Ledkov
    File: debian/patches/shutdown-do-not-detach-autoclear-loopback-devices.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3978d34b59e98cdd01836c41a10442967636b8fc

  [ Balint Reczey ]
  * Revert upstream commit breaking IPv4 DHCP in LXC containers in 244.1
    (LP: #1857123)
    File: debian/patches/Revert-network-if-sys-is-rw-then-udev-should-be-around.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=39c12f8e736afd1b7bdeb13ff6bccaea85020873

systemd (244.1-0ubuntu1) focal; urgency=medium

  * New upstream version 244.1
    - network: set ipv6 mtu after link-up or device mtu change (LP: #1671951)
    - & other changes
  * Refresh patches.
    - Dropped changes:
      * d/p/lp-1853852-*: fix issues with muliplexed shmat calls (LP: #1853852)
        Files:
        - debian/patches/lp-1853852-seccomp-fix-multiplexed-system-calls.patch
        - debian/patches/lp-1853852-seccomp-mmap-test-results-depend-on-kernel-libseccom.patch
        https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=382271662c60c339b0a404c7a1772fe5670516ef
      * d/p/lp1671951-network-set-ipv6-mtu-after-link-up-or-device-mtu-cha.patch:
        set ipv6 mtu at correct time
  * pstore: Don't start systemd-pstore.service in containers.
    Usually it is not useful and can also fail making
    boot-and-services autopkgtest fail. (LP: #1856729)
    File: debian/patches/pstore-Don-t-start-systemd-pstore.service-in-containers.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=28b5a03769cbed9d3170ebac38508b867530a2d6
  * Revert: network: do not drop foreign config if interface is in initialized state.
    This fixes FTBFS with the other network-related reverts.
    File: debian/patches/Revert-network-do-not-drop-foreign-config-if-interface-is.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=22a9fa3bb03ba2a629926af39ea7df81fe33c9b8

systemd (244-3ubuntu5) focal; urgency=medium

  [ Dariusz Gadomski ]
  * d/p/lp1762391/0001-user-util-Add-helper-functions-for-gid-lists-operati.patch,
    d/p/lp1762391/0002-execute-Restore-call-to-pam_setcred.patch,
    d/p/lp1762391/0003-execute-Detect-groups-added-by-PAM-and-merge-them-wi.patch,
    d/p/lp1762391/0004-test-Add-tests-for-gid-list-ops.patch,
    d/p/lp1762391/0005-execute-add-const-to-array-parameters-where-possible.patch,
    d/p/lp1762391/0006-execute-allow-pam_setcred-to-fail-ignore-errors.patch:
    - Restore call to pam_setcred (LP: #1762391)

  [ Dan Streetman ]
  * d/t/storage: without scsi_debug, skip test (LP: #1847816)

systemd (244-3ubuntu4) focal; urgency=medium

  * d/p/lp1671951-network-set-ipv6-mtu-after-link-up-or-device-mtu-cha.patch:
    set ipv6 mtu at correct time (LP: #1671951)
  * d/p/0001-network-rename-linux_configure_after_setting_mtu-to-linux.patch,
    d/p/0002-network-add-link-setting_genmode-flag.patc...

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted systemd into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/242-7ubuntu3.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.34 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (237-3ubuntu10.34) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

php7.2/7.2.24-0ubuntu0.18.04.2 (armhf)
openssh/1:7.6p1-4ubuntu0.3 (arm64, armhf, ppc64el, amd64, s390x, i386)
dovecot/1:2.2.33.2-1ubuntu4.5 (armhf)
gvfs/1.36.1-0ubuntu1.3.3 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (242-7ubuntu3.3) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

samba/2:4.10.7+dfsg-0ubuntu2.4 (armhf)
netplan.io/0.98-0ubuntu1 (amd64, ppc64el)
gnome-desktop3/3.34.2-2ubuntu1~19.10.1 (armhf)
systemd/242-7ubuntu3.3 (ppc64el, arm64)
munin/2.0.49-3ubuntu1 (armhf)
bolt/0.8-4 (armhf)
umockdev/0.13.2-1 (armhf)
openssh/1:8.0p1-6build1 (amd64, ppc64el, i386, s390x, arm64, armhf)
linux-oem-osp1/5.0.0-1037.42 (amd64)
multipath-tools/unknown (armhf)
knot-resolver/3.2.1-3 (amd64, ppc64el)
lxc/3.0.4-0ubuntu1 (amd64, ppc64el, i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted systemd into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/242-7ubuntu3.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.39 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (237-3ubuntu10.39) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

php7.2/7.2.24-0ubuntu0.18.04.2 (armhf)
gvfs/1.36.1-0ubuntu1.3.3 (ppc64el)
lxc/3.0.3-0ubuntu1~18.04.1 (amd64)
systemd/237-3ubuntu10.39 (i386)
netplan.io/0.98-0ubuntu1~18.04.1 (i386, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (242-7ubuntu3.7) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

remctl/3.15-1build2 (armhf)
systemd-bootchart/unknown (armhf)
netplan.io/0.98-0ubuntu1 (amd64)
systemd/242-7ubuntu3.7 (ppc64el, s390x)
sks/unknown (armhf)
munin/2.0.49-3ubuntu1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Dan Streetman (ddstreet)]

'storage' autopkgtest now passes (is skipped) when testing with linux-kvm:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-eoan/eoan/amd64/s/systemd/20200210_144351_da247@/log.gz
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/s/systemd/20200210_151352_6b64a@/log.gz

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 242-7ubuntu3.7

---------------
systemd (242-7ubuntu3.7) eoan; urgency=medium

  [ Dariusz Gadomski ]
  * d/p/lp1762391/0001-Call-getgroups-to-know-size-of-supplementary-groups-.patch,
    d/p/lp1762391/0002-user-util-tweak-to-in_gid.patch,
    d/p/lp1762391/0003-user-util-Add-helper-functions-for-gid-lists-operati.patch,
    d/p/lp1762391/0004-execute-Restore-call-to-pam_setcred.patch,
    d/p/lp1762391/0005-execute-Detect-groups-added-by-PAM-and-merge-them-wi.patch,
    d/p/lp1762391/0006-test-Add-tests-for-gid-list-ops.patch,
    d/p/lp1762391/0007-execute-add-const-to-array-parameters-where-possible.patch,
    d/p/lp1762391/0008-execute-allow-pam_setcred-to-fail-ignore-errors.patch:
    - Restore call to pam_setcred (LP: #1762391)

  * d/p/lp1846232/0001-network-honor-MTUBytes-setting.patch,
    d/p/lp1846232/0002-network-bump-MTU-bytes-only-when-MTUByte-is-not-set.patch:
    - do not always bump MTU with additional 4bytes (LP: #1846232)
  * d/p/lp1671951-network-set-ipv6-mtu-after-link-up-or-device-mtu-cha.patch:
    - set ipv6 mtu at correct time (LP: #1671951)
  * d/p/lp1845909/0001-network-rename-linux_configure_after_setting_mtu-to-linux.patch,
    d/p/lp1845909/0002-network-add-link-setting_genmode-flag.patch,
    d/p/lp1845909/0003-network-if-ipv6ll-is-disabled-enumerate-tentative-ipv6-ad.patch,
    d/p/lp1845909/0004-network-drop-foreign-config-after-addr_gen_mode-has-been-.patch,
    d/p/lp1845909/0005-network-drop-IPv6LL-address-when-LinkLocalAddressing.patch:
    - drop foreign config and raise interface after setting genmode
      (LP: #1845909)
  * d/t/storage: without scsi_debug, skip test (LP: #1847816)

 -- Dan Streetman <email address hidden> Thu, 06 Feb 2020 09:45:57 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 237-3ubuntu10.39

---------------
systemd (237-3ubuntu10.39) bionic; urgency=medium

  [ Dariusz Gadomski ]
  * d/p/lp1762391/0001-Call-getgroups-to-know-size-of-supplementary-groups-.patch,
    d/p/lp1762391/0002-user-util-tweak-to-in_gid.patch,
    d/p/lp1762391/0003-user-util-Add-helper-functions-for-gid-lists-operati.patch,
    d/p/lp1762391/0004-execute-Restore-call-to-pam_setcred.patch,
    d/p/lp1762391/0005-execute-Detect-groups-added-by-PAM-and-merge-them-wi.patch,
    d/p/lp1762391/0006-test-Add-tests-for-gid-list-ops.patch,
    d/p/lp1762391/0007-execute-add-const-to-array-parameters-where-possible.patch,
    d/p/lp1762391/0008-execute-allow-pam_setcred-to-fail-ignore-errors.patch:
    - Restore call to pam_setcred (LP: #1762391)

  [ Ioanna Alifieraki ]
  * d/p/lp1860548/0001-Revert-Replace-use-of-snprintf-with-xsprintf.patch,
    d/p/lp1860548/0002-job-truncate-unit-description.patch:
    - use snprintf instead of xsprintf (LP: #1860548)

  [ Dan Streetman ]
  * d/p/lp1833193-network-update-address-when-static-address-was-alrea.patch:
    - Update lft when static addr was cfg by dhcp (LP: #1833193)
  * d/p/lp1849261/0001-core-when-we-can-t-enqueue-OnFailure-job-show-full-e.patch,
    d/p/lp1849261/0002-core-don-t-trigger-OnFailure-deps-when-a-unit-is-goi.patch:
    - Only trigger OnFailure= if Restart= is not in effect (LP: #1849261)
  * d/p/lp1671951-network-set-ipv6-mtu-after-link-up-or-device-mtu-cha.patch:
    - set ipv6 mtu at correct time (LP: #1671951)
  * d/p/lp1845909/0001-networkd-honour-LinkLocalAddressing.patch,
    d/p/lp1845909/0002-networkd-fix-link_up-12505.patch,
    d/p/lp1845909/0003-network-do-not-send-ipv6-token-to-kernel.patch,
    d/p/lp1845909/0004-network-rename-linux_configure_after_setting_mtu-to-linux.patch,
    d/p/lp1845909/0005-network-add-link-setting_genmode-flag.patch,
    d/p/lp1845909/0006-network-if-ipv6ll-is-disabled-enumerate-tentative-ipv6-ad.patch,
    d/p/lp1845909/0007-network-drop-foreign-config-after-addr_gen_mode-has-been-.patch,
    d/p/lp1845909/0008-network-drop-IPv6LL-address-when-LinkLocalAddressing.patch:
    - if LinkLocalAddressing=no prevent creation of ipv6ll (LP: #1845909)
  * d/p/lp1859862-network-Do-not-disable-IPv6-by-writing-to-sysctl.patch:
    - enable ipv6 when needed (LP: #1859862)
  * d/p/lp1836695-networkd-Add-back-static-routes-after-DHCPv4-lease-e.patch:
    - (re)add static routes after getting dhcp4 addr (LP: #1836695)
  * d/t/storage:
    - fix buggy test (LP: #1831459)
    - without scsi_debug, skip test (LP: #1847816)

 -- Dan Streetman <email address hidden> Thu, 06 Feb 2020 10:00:49 -0500

[Łukasz Zemczak (sil2100)]

Hello Dan, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu21.28 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (229-4ubuntu21.28) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

udisks2/2.1.7-1ubuntu1 (amd64)
multipath-tools/0.5.0+git1.656f8865-5ubuntu2.5 (armhf)
nplan/0.32~16.04.7 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Dan Streetman (ddstreet)]

storage autopkgtest now passes on xenial.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 229-4ubuntu21.28

---------------
systemd (229-4ubuntu21.28) xenial; urgency=medium

  * d/t/logind: skip if nonexistent /sys/power/state (LP: #1862657)
  * d/p/lp1839290-Change-job-mode-of-manager-triggered-restarts-to-JOB.patch:
    - when restarting service after failure, replace existing queued jobs
      (LP: #1839290)
  * d/t/storage:
    - fix buggy test (LP: #1831459)
    - without scsi_debug, skip test (LP: #1847816)
  * d/p/lp1867421-70-mouse.hwdb-Set-DPI-for-MS-Classic-IntelliMouse.patch:
    - fix resolution of IntelliMouse (LP: #1867421)
  * d/p/lp1858412-journalctl-allow-running-vacuum-on-remote-journals-t.patch:
    - allow vacuuming journal 'root' dir (LP: #1858412)
  * d/p/lp1862232-network-DHCP-ignore-error-in-setting-hostname-when-i.patch:
    - do not let invalid hostname break dhcpv4 (LP: #1862232)
  * d/t/systemd-fsckd: Skip test on arm64 (LP: #1870194)
  * d/p/lp1837914-journal-do-not-trigger-assertion-when-journal_file_c.patch:
    - do not crash if NULL passted to journal destructor (LP: #1837914)
  * d/e/initramfs-tools/hooks/udev:
    - Follow symlinks when finding link files to copy into initramfs
      (LP: #1868892)

 -- Dan Streetman <email address hidden> Mon, 20 Apr 2020 10:17:40 -0400