Activity log for bug #2045586

Date Who What changed Old value New value Message
2023-12-04 19:35:25 Steve Langasek bug added bug
2023-12-04 19:36:23 Steve Langasek bug task added livecd-rootfs (Ubuntu)
2023-12-04 19:36:30 Steve Langasek bug task added linux (Ubuntu)
2024-01-23 16:40:54 dann frazier summary livecd-rootfs uses losetup -P for theoretically reliable/synchronous partition setup but it's not reliable in noble livecd-rootfs uses losetup -P for theoretically reliable/synchronous partition setup but it's not reliable
2024-01-24 18:48:07 Launchpad Janitor merge proposal linked https://code.launchpad.net/~mwhudson/livecd-rootfs/+git/livecd-rootfs/+merge/459380
2024-01-25 04:23:46 Launchpad Janitor livecd-rootfs (Ubuntu): status New Fix Released
2024-01-31 19:17:19 Launchpad Janitor merge proposal linked https://code.launchpad.net/~dannf/livecd-rootfs/+git/livecd-rootfs/+merge/459549
2024-02-03 16:14:46 dann frazier nominated for series Ubuntu Mantic
2024-02-03 16:14:46 dann frazier bug task added util-linux (Ubuntu Mantic)
2024-02-03 16:14:46 dann frazier bug task added livecd-rootfs (Ubuntu Mantic)
2024-02-03 16:14:46 dann frazier bug task added linux (Ubuntu Mantic)
2024-02-03 16:14:46 dann frazier nominated for series Ubuntu Jammy
2024-02-03 16:14:46 dann frazier bug task added util-linux (Ubuntu Jammy)
2024-02-03 16:14:46 dann frazier bug task added livecd-rootfs (Ubuntu Jammy)
2024-02-03 16:14:46 dann frazier bug task added linux (Ubuntu Jammy)
2024-02-03 16:14:46 dann frazier nominated for series Ubuntu Noble
2024-02-03 16:14:46 dann frazier bug task added util-linux (Ubuntu Noble)
2024-02-03 16:14:46 dann frazier bug task added livecd-rootfs (Ubuntu Noble)
2024-02-03 16:14:46 dann frazier bug task added linux (Ubuntu Noble)
2024-02-09 15:22:31 Catherine Redfield bug added subscriber Catherine Redfield
2024-02-18 17:57:52 Launchpad Janitor merge proposal linked https://code.launchpad.net/~mwhudson/livecd-rootfs/+git/livecd-rootfs/+merge/460729
2024-02-18 20:18:53 Launchpad Janitor merge proposal linked https://code.launchpad.net/~mwhudson/livecd-rootfs/+git/livecd-rootfs/+merge/460732
2024-02-18 20:24:45 Michael Hudson-Doyle description In mantic, we migrated livecd-rootfs to use losetup -P instead of kpartx, with the expectation that this would give us a reliable, race-free way of loop-mounting partitions from a disk image during image build. In noble, we are finding that it is no longer reliable, and in fact fails rather often. It is most noticeable with riscv64 builds, which is the architecture where we most frequently ran into problems before with kpartx. The first riscv64+generic build in noble where the expected loop partition device is not available is https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/noble/cpc/+build/531790 The failure is however not unique to riscv64, and the autopkgtest for the latest version of livecd-rootfs (24.04.7) - an update that specifically tries to add more debugging code for this scenario - has also failed on ppc64el. https://autopkgtest.ubuntu.com/packages/l/livecd-rootfs/noble/ppc64el The first failure happened on November 16. While there has been an update to the util-linux package in noble, this did not land until November 23. [impact] In mantic, we migrated livecd-rootfs to use losetup -P instead of kpartx, with the expectation that this would give us a reliable, race-free way of loop-mounting partitions from a disk image during image build. In noble, we are finding that it is no longer reliable, and in fact fails rather often. It is most noticeable with riscv64 builds, which is the architecture where we most frequently ran into problems before with kpartx. The first riscv64+generic build in noble where the expected loop partition device is not available is   https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/noble/cpc/+build/531790 The failure is however not unique to riscv64, and the autopkgtest for the latest version of livecd-rootfs (24.04.7) - an update that specifically tries to add more debugging code for this scenario - has also failed on ppc64el.   https://autopkgtest.ubuntu.com/packages/l/livecd-rootfs/noble/ppc64el The first failure happened on November 16. While there has been an update to the util-linux package in noble, this did not land until November 23. The losetup usage has been backported to Jammy, and sees frequent failures there. [test case] The autopkgtests will provide enough confidence that the changes are not completely broken. Whether the change helps with the races on riscv can be "tested in prod" just as well as any other way. [regression potential] If the backport has been done incorrectly, image builds can fail (and the autopkgtests will fail if it has been completely bungled). This can be quickly handled. There is no foreseeable way for this to result in successful builds but broken images, which would be a much more difficult failure mode to unpick.
2024-02-18 21:43:11 Ubuntu Archive Robot bug added subscriber Michael Hudson-Doyle
2024-02-19 16:08:01 Łukasz Zemczak livecd-rootfs (Ubuntu Jammy): status New Fix Committed
2024-02-19 16:08:03 Łukasz Zemczak bug added subscriber Ubuntu Stable Release Updates Team
2024-02-19 16:08:06 Łukasz Zemczak bug added subscriber SRU Verification
2024-02-19 16:08:09 Łukasz Zemczak tags verification-needed verification-needed-jammy
2024-02-19 18:15:59 Launchpad Janitor livecd-rootfs (Ubuntu Jammy): status Fix Committed Fix Released
2024-02-20 08:54:57 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ubuntu-core-dev/livecd-rootfs/+git/livecd-rootfs/+merge/460810
2024-04-29 22:37:07 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/livecd-rootfs/+git/livecd-rootfs/+merge/465193