ppcel64 images unpack to /boot/vmlinux not /boot/vmlinuz leading to kernel feature mismatch

[John Chittum]

In a build of a ppcel64 image, snap_validate_seed fails to find the installed kernel to pick a matching apparmor feature set to mount. from the log

```
+ [ -e chroot/boot/vmlinuz ]
+ [ -e chroot/var/lib/snapd/seed/seed.yaml ]
+ snap debug validate-seed chroot/var/lib/snapd/seed/seed.yaml
+ realpath chroot
+ /usr/lib/snapd/snap-preseed --reset /build/chroot
ubuntu classic preseeding
unmounting: /build/chroot/tmp/snapd-preseed
ubuntu classic preseeding
unmounting: /tmp/snapd-preseed
+ realpath chroot
+ /usr/lib/snapd/snap-preseed /build/chroot
ubuntu classic preseeding
```

This corresponds to line 772 in livecd-rootfs/live-build/functions

snap_validate_seed() {
    local CHROOT_ROOT=$1
    local kern_major_min=undefined

    if [ -e ${CHROOT_ROOT}/boot/vmlinuz ]; then

ppcel64 however unpacks to /boot/vmlinux. For history:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1358920

for ppcel64 images, this function needs to be updated to check for `/boot/vmlinux`.

This will require an SRU 23.10 (post-release) and 22.04

SRU
====

[ Impact ]

 * ppc64el Users of the 6.5 kernel will have un-optimized first boot experiences due to snaps not preseeding with the correct apparmor setup. This directly affects clouds, leading to boot speed degradation of anywhere from 10-30s (depending on snaps installed)

[ Test Plan ]

 * Build test ppc64el images and test that preseeding was correctly applied for the 6.5 kernel

[ Where problems could occur ]

 * If the optimized preseeding of the snaps for ppc64el continues to fail

[Philip Roche (philroche)]

We haven't encountered this issue on non mantic cloud images but this should be backported to earlier branches/releases.

[Philip Roche (philroche)]

MP to master/devel @ https://code.launchpad.net/~philroche/livecd-rootfs/+git/livecd-rootfs/+merge/453306

Backport to mantic @ https://code.launchpad.net/~philroche/livecd-rootfs/+git/livecd-rootfs/+merge/454045

[Philip Roche (philroche)]

Backport to Jammy @ https://code.launchpad.net/~philroche/livecd-rootfs/+git/livecd-rootfs/+merge/454052

[Philip Roche (philroche)]

As per kernel lifecycle @ https://ubuntu.com/kernel/lifecycle lunar, focal, bionic and xenial will not be getting any major kernel version bumps so this bug will not affect those series. As such I will mark these as Won't Fix

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 24.04.2

---------------
livecd-rootfs (24.04.2) noble; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

 -- Philip Roche <email address hidden> Thu, 26 Oct 2023 17:56:00 +0100

[Utkarsh Gupta (utkarsh)]

Please adjust the status of Noble, too.

[Steve Langasek (vorlon)]

"adjust the status of noble too"? the status showed that this change landed in noble.

[Steve Langasek (vorlon)]

An upload of livecd-rootfs to mantic-proposed has been rejected from the upload queue for the following reason: "cruft in the upload (livecd-rootfs-23.10.56/arm64-minimized-alignment.patch, .idea)".

[Steve Langasek (vorlon)]

An upload of livecd-rootfs to jammy-proposed has been rejected from the upload queue for the following reason: "cruft in upload (.idea*/)".

[Philip Roche (philroche)]

Apologies RE jammy upload. I have now re-uploaded Jammy livecd-rootfs 2.765.30 again without any '.idea' directory cruft.

[Philip Roche (philroche)]

Apologies RE mantic upload. I have now re-uploaded Mantic livecd-rootfs 23.10.56 again without any '.idea' directory cruft.

[Steve Langasek (vorlon)]

Hello John, or anyone else affected,

Accepted livecd-rootfs into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/23.10.56 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted livecd-rootfs (23.10.56) for mantic have finished running.
The following regressions have been reported in tests triggered by the package:

livecd-rootfs/23.10.56 (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/mantic/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Philip Roche (philroche)]

I retriggered the s390x tests and they did pass https://autopkgtest.ubuntu.com/packages/l/livecd-rootfs/mantic/s390x

[Timo Aaltonen (tjaalton)]

missing verification

[Andreas Hasenack (ahasenack)]

This is missing verification, and there is a new 23.10.57 upload in mantic unapproved which does not include this change in its .changes file.

[Andreas Hasenack (ahasenack)]

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.32 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[John Chittum (jchittum)]

Mantic build verified

1. created cpc livecd-rootfs package in my personal PPA
(not public, viewable if you're in ~cloudware)
https://launchpad.net/~cloudware/+archive/ubuntu/secret-sauce-jchittum/+packages

2. launched a mantic build
(not public, but someone in ~cloudware can see: https://launchpad.net/~cloudware/+livefs/ubuntu/mantic/cpc-development/+build/543950 )

3. checked for matching `vmlinux` and not `vmlinuz`
+ [ -e chroot/boot/vmlinux ]
+ boot_filename=vmlinux

[John Chittum (jchittum)]

Jammy build verified

1. created cpc livecd-rootfs package in my personal PPA
(not public, viewable if you're in ~cloudware)
https://launchpad.net/~cloudware/+archive/ubuntu/secret-sauce-jchittum/+packages

2. launched a mantic build
(not public, but someone in ~cloudware can see: https://launchpad.net/~cloudware/+livefs/ubuntu/jammy/cpc-development/+build/543951 )

3. checked for matching `vmlinux` and not `vmlinuz`
+ [ -e chroot/boot/vmlinux ]
+ boot_filename=vmlinux

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 23.10.56

---------------
livecd-rootfs (23.10.56) mantic; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

 -- Philip Roche <email address hidden> Wed, 11 Oct 2023 11:41:00 +0100

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Łukasz Zemczak (sil2100)]

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.33 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.34 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Philip Roche (philroche)]

This bug was verified for Jammy in https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/2038957/comments/19 and the area of changes in the uploads since then does not affect this bug. As such I am marking as verification done again.

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 2.765.34

---------------
livecd-rootfs (2.765.34) jammy; urgency=medium

  * Remove comments in the debian/control dependencies as it was causing the
    package build to fail.

livecd-rootfs (2.765.33) jammy; urgency=medium

  [ Steve Langasek ]
  * Use losetup instead of kpartx to resolve race conditions in riscv64
    image builds. LP: #2045797.

  [ Utkarsh Gupta ]
  * unminimize: Use lxd-installer to install LXD itself (LP: #2036725)

livecd-rootfs (2.765.32) jammy; urgency=medium

  [ Heinrich Schuchardt ]
  * arm: fix console parameter for ARM cloud-images (LP: #2036730)

livecd-rootfs (2.765.31) jammy; urgency=medium

  [ John Chittum ]
  * fix: add 6.5 kernel apparmor features to livecd-rootfs based on
    features of 6.5 in ubuntu/mantic. This will roll as HWE. (LP: #2037567)

livecd-rootfs (2.765.30) jammy; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

livecd-rootfs (2.765.29) jammy; urgency=medium

  [ Steve Langasek ]
  * The chroot tmpfs mount should only be /var/lib/apt/lists, not
    /var/lib/apt; the latter breaks changes to /var/lib/apt/extended_states.
    (LP: #2036195).

livecd-rootfs (2.765.28) jammy; urgency=medium

  * Fix unminimize to correctly list packages. (LP: #1996489)
  * Install LXD snap from stable/ubuntu-<version> channel. (LP: #2036725)

livecd-rootfs (2.765.27) jammy; urgency=medium

  [ Thomas Bechtold ]
  * Do not modify /etc/ssh/sshd_config for ubuntu-cpc
    project builds (LP: #1968873)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Mon, 15 Jan 2024 16:41:18 +0100