Activity log for bug #2028862

Date Who What changed Old value New value Message
2023-07-27 13:48:54 Frank Heimes bug added bug
2023-07-27 13:49:12 Frank Heimes bug task added snapd
2023-07-27 13:49:28 Frank Heimes summary Mantic hands during install with continuous 'Job snapd.seeded.service/start running' messages Mantic hangs during install with continuous 'Job snapd.seeded.service/start running' messages
2023-07-27 13:50:15 Frank Heimes bug task added ubuntu-z-systems
2023-07-27 13:50:39 Frank Heimes description While trying to install the latest mantic ISO image (tried current and pending) on s390x (but I think this is not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ... While trying to install the latest mantic ISO image (tried current and pending) on s390x (but it's probably not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ...
2023-07-27 14:07:27 Dan Bungert subiquity: status New Invalid
2023-08-02 08:59:13 Frank Heimes ubuntu-z-systems: importance Undecided High
2023-08-02 08:59:57 Frank Heimes ubuntu-z-systems: assignee Snappy Developers (snappy-dev)
2023-08-04 16:10:04 Heinrich Schuchardt attachment added Syslog mantic riscv64 installer with 180 s timeout for snapd.seeded.service https://bugs.launchpad.net/subiquity/+bug/2028862/+attachment/5690337/+files/syslog-2023-08-04-18-00.log.gz
2023-08-08 00:27:41 Dan Bungert subiquity: status Invalid New
2023-08-08 01:01:56 Michael Hudson-Doyle bug task added livecd-rootfs (Ubuntu)
2023-08-08 01:12:48 Launchpad Janitor merge proposal linked https://code.launchpad.net/~mwhudson/livecd-rootfs/+git/livecd-rootfs/+merge/448655
2023-08-08 01:13:37 Michael Hudson-Doyle tags mantic foundations-todo mantic
2023-08-08 01:13:41 Michael Hudson-Doyle livecd-rootfs (Ubuntu): assignee Michael Hudson-Doyle (mwhudson)
2023-08-08 03:34:27 Michael Hudson-Doyle livecd-rootfs (Ubuntu): status New In Progress
2023-08-08 03:34:34 Michael Hudson-Doyle subiquity: status New In Progress
2023-08-08 03:34:39 Michael Hudson-Doyle subiquity: assignee Michael Hudson-Doyle (mwhudson)
2023-08-08 06:09:30 Frank Heimes ubuntu-z-systems: status New In Progress
2023-08-08 16:54:10 Launchpad Janitor livecd-rootfs (Ubuntu): status In Progress Fix Released
2023-08-16 19:34:31 Dan Bungert snapd: status New Invalid
2023-08-16 19:34:38 Dan Bungert subiquity: status In Progress Invalid
2023-08-18 10:38:02 Frank Heimes ubuntu-z-systems: status In Progress Fix Released
2023-08-28 16:39:00 Dan Bungert livecd-rootfs (Ubuntu): milestone ubuntu-22.04.4
2023-08-28 16:39:24 Dan Bungert livecd-rootfs (Ubuntu): milestone ubuntu-22.04.4
2023-08-28 16:40:13 Dan Bungert nominated for series Ubuntu Jammy
2023-08-28 16:40:13 Dan Bungert bug task added livecd-rootfs (Ubuntu Jammy)
2023-08-28 16:40:27 Dan Bungert livecd-rootfs (Ubuntu Jammy): assignee Dan Bungert (dbungert)
2023-08-28 20:04:52 Dan Bungert description While trying to install the latest mantic ISO image (tried current and pending) on s390x (but it's probably not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ... [ Impact ] * The subiquity systemd units and cloud-init 23.3 have an incompatibility that results in a stuck boot, when cloud-init 23.3 is present on an unpatched install ISO. * This is a necessary prerequisite to allow for Jammy dailies / eventually 22.04.4 with updated cloud-init. [ Test Plan ] * Live-server Daily ISOs constructed with the fixed version should boot correctly to the Subqiuity TUI, regardless of the cloud-init version present in that ISO. * If desired, a test plan with ~16 steps can be provided with detailed steps to modify ISOs to simulate the bug and the fix. [ Where problems could occur ] * Changes to livecd-rootfs cause frequent regressions to other build projects. The risk here is relatively localized to live-server builds as we are only modifying drop-in systemd files in use by live-server. * If this change is done incorrectly, when cloud-init 23.3 is SRUed and allowed to migrate, the boot of the live-server dailies may show the same problem as the original bug. * "The livecd-rootfs package is a frequent target of SRUs as part of development of changes to image builds for the target series, and is not intended for general installation on end-user systems. The risk of user-affecting regression is lower as a result, because the impact of changes to this package to end users is mediated by way of image builds." https://wiki.ubuntu.com/StableReleaseUpdates?action=show&redirect=StableReleaseUpdate#livecd-rootfs [ Other Info ] * Using kernel command line "systemd.mask=snapd.seeded.service" allows affected ISOs to boot successfully --- previous description: While trying to install the latest mantic ISO image (tried current and pending) on s390x (but it's probably not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ...
2023-08-28 20:10:01 Dan Bungert description [ Impact ] * The subiquity systemd units and cloud-init 23.3 have an incompatibility that results in a stuck boot, when cloud-init 23.3 is present on an unpatched install ISO. * This is a necessary prerequisite to allow for Jammy dailies / eventually 22.04.4 with updated cloud-init. [ Test Plan ] * Live-server Daily ISOs constructed with the fixed version should boot correctly to the Subqiuity TUI, regardless of the cloud-init version present in that ISO. * If desired, a test plan with ~16 steps can be provided with detailed steps to modify ISOs to simulate the bug and the fix. [ Where problems could occur ] * Changes to livecd-rootfs cause frequent regressions to other build projects. The risk here is relatively localized to live-server builds as we are only modifying drop-in systemd files in use by live-server. * If this change is done incorrectly, when cloud-init 23.3 is SRUed and allowed to migrate, the boot of the live-server dailies may show the same problem as the original bug. * "The livecd-rootfs package is a frequent target of SRUs as part of development of changes to image builds for the target series, and is not intended for general installation on end-user systems. The risk of user-affecting regression is lower as a result, because the impact of changes to this package to end users is mediated by way of image builds." https://wiki.ubuntu.com/StableReleaseUpdates?action=show&redirect=StableReleaseUpdate#livecd-rootfs [ Other Info ] * Using kernel command line "systemd.mask=snapd.seeded.service" allows affected ISOs to boot successfully --- previous description: While trying to install the latest mantic ISO image (tried current and pending) on s390x (but it's probably not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ... [ Impact ] * The subiquity systemd units and cloud-init 23.3 have an incompatibility that results in a stuck boot, when cloud-init 23.3 is present on an unpatched install ISO. The symptom looks like the following: "A start job is waiting for Wait until snapd is fully seeded" This will never complete.  * This is a necessary prerequisite to allow for Jammy dailies /    eventually 22.04.4 with updated cloud-init. [ Test Plan ]  * Live-server Daily ISOs constructed with the fixed version    should boot correctly to the Subqiuity TUI, regardless of the    cloud-init version present in that ISO.  * If desired, a test plan with ~16 steps can be provided with    detailed steps to modify ISOs to simulate the bug and the fix. [ Where problems could occur ]  * Changes to livecd-rootfs cause frequent regressions to other    build projects. The risk here is relatively localized to    live-server builds as we are only modifying drop-in systemd    files in use by live-server.  * If this change is done incorrectly, when cloud-init 23.3 is    SRUed and allowed to migrate, the boot of the live-server    dailies may show the same problem as the original bug.  * "The livecd-rootfs package is a frequent target of SRUs as part    of development of changes to image builds for the target    series, and is not intended for general installation on    end-user systems. The risk of user-affecting regression is    lower as a result, because the impact of changes to this    package to end users is mediated by way of image builds." https://wiki.ubuntu.com/StableReleaseUpdates?action=show&redirect=StableReleaseUpdate#livecd-rootfs [ Other Info ]  * Using kernel command line "systemd.mask=snapd.seeded.service"    allows affected ISOs to boot successfully --- previous description: While trying to install the latest mantic ISO image (tried current and pending) on s390x (but it's probably not limited to a certain architecture) the installation hangs with never-ending messages like this: M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 23s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 24s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 25s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 26s / no limit) M[K[ [0;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m] Job snapd.seeded.service/start running (16min 27s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 28s / no limit) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 29s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 30s / no limit) M[K[[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no limit) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] Job snapd.seeded.service/start running (16min 31s / no (sorry for the 'special' characters, but it's because I copied the content from the HMC console) I was told that using the kernel arg "systemd.mask=snapd.seeded.service" should help to workaround this, still need to try this out ...
2023-08-28 20:17:46 Launchpad Janitor merge proposal linked https://code.launchpad.net/~dbungert/livecd-rootfs/+git/livecd-rootfs/+merge/450069
2023-08-28 20:22:33 Dan Bungert bug added subscriber Dan Bungert
2023-08-28 20:23:33 Dan Bungert livecd-rootfs (Ubuntu Jammy): status New In Progress
2023-09-02 02:58:28 Steve Langasek livecd-rootfs (Ubuntu Jammy): status In Progress Fix Committed
2023-09-02 02:58:29 Steve Langasek bug added subscriber Ubuntu Stable Release Updates Team
2023-09-02 02:58:31 Steve Langasek bug added subscriber SRU Verification
2023-09-02 02:58:34 Steve Langasek tags foundations-todo mantic foundations-todo mantic verification-needed verification-needed-jammy
2023-09-05 21:22:16 Dan Bungert tags foundations-todo mantic verification-needed verification-needed-jammy foundations-todo mantic verification-done verification-done-jammy
2023-09-21 20:30:24 Launchpad Janitor livecd-rootfs (Ubuntu Jammy): status Fix Committed Fix Released
2023-09-21 20:30:28 Andreas Hasenack removed subscriber Ubuntu Stable Release Updates Team
2023-10-23 14:56:42 Benjamin Drung tags foundations-todo mantic verification-done verification-done-jammy mantic verification-done verification-done-jammy