ppcel64 images unpack to /boot/vmlinux not /boot/vmlinuz leading to kernel feature mismatch

Bug #2038957 reported by John Chittum
24
This bug affects 2 people
Affects Status Importance Assigned to Milestone
livecd-rootfs (Ubuntu)
Fix Released
Undecided
Philip Roche
Xenial
Won't Fix
Undecided
Philip Roche
Bionic
Won't Fix
Undecided
Philip Roche
Focal
Won't Fix
Undecided
Philip Roche
Jammy
Fix Released
Undecided
Philip Roche
Lunar
Won't Fix
Undecided
Philip Roche
Mantic
Fix Released
Undecided
Philip Roche
Noble
Fix Released
Undecided
Philip Roche

Bug Description

In a build of a ppcel64 image, snap_validate_seed fails to find the installed kernel to pick a matching apparmor feature set to mount. from the log

```
+ [ -e chroot/boot/vmlinuz ]
+ [ -e chroot/var/lib/snapd/seed/seed.yaml ]
+ snap debug validate-seed chroot/var/lib/snapd/seed/seed.yaml
+ realpath chroot
+ /usr/lib/snapd/snap-preseed --reset /build/chroot
ubuntu classic preseeding
unmounting: /build/chroot/tmp/snapd-preseed
ubuntu classic preseeding
unmounting: /tmp/snapd-preseed
+ realpath chroot
+ /usr/lib/snapd/snap-preseed /build/chroot
ubuntu classic preseeding
```

This corresponds to line 772 in livecd-rootfs/live-build/functions

snap_validate_seed() {
    local CHROOT_ROOT=$1
    local kern_major_min=undefined

    if [ -e ${CHROOT_ROOT}/boot/vmlinuz ]; then

ppcel64 however unpacks to /boot/vmlinux. For history:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1358920

for ppcel64 images, this function needs to be updated to check for `/boot/vmlinux`.

This will require an SRU 23.10 (post-release) and 22.04

SRU
====

[ Impact ]

 * ppc64el Users of the 6.5 kernel will have un-optimized first boot experiences due to snaps not preseeding with the correct apparmor setup. This directly affects clouds, leading to boot speed degradation of anywhere from 10-30s (depending on snaps installed)

[ Test Plan ]

 * Build test ppc64el images and test that preseeding was correctly applied for the 6.5 kernel

[ Where problems could occur ]

 * If the optimized preseeding of the snaps for ppc64el continues to fail

Related branches

Philip Roche (philroche)
summary: - ppcel64 images unpack to /boot/vmlinux not /boot/vmlinux leading to
+ ppcel64 images unpack to /boot/vmlinux not /boot/vmlinuz leading to
kernel feature mismatch
Changed in livecd-rootfs (Ubuntu):
assignee: nobody → Philip Roche (philroche)
Revision history for this message
Philip Roche (philroche) wrote :

We haven't encountered this issue on non mantic cloud images but this should be backported to earlier branches/releases.

Changed in livecd-rootfs (Ubuntu):
status: New → Fix Committed
Philip Roche (philroche)
description: updated
Revision history for this message
Philip Roche (philroche) wrote :
Revision history for this message
Philip Roche (philroche) wrote :
Changed in livecd-rootfs (Ubuntu Jammy):
status: New → Fix Committed
assignee: nobody → Philip Roche (philroche)
Revision history for this message
Philip Roche (philroche) wrote :

As per kernel lifecycle @ https://ubuntu.com/kernel/lifecycle lunar, focal, bionic and xenial will not be getting any major kernel version bumps so this bug will not affect those series. As such I will mark these as Won't Fix

Changed in livecd-rootfs (Ubuntu Lunar):
assignee: nobody → Philip Roche (philroche)
Changed in livecd-rootfs (Ubuntu Focal):
assignee: nobody → Philip Roche (philroche)
Changed in livecd-rootfs (Ubuntu Bionic):
assignee: nobody → Philip Roche (philroche)
Changed in livecd-rootfs (Ubuntu Xenial):
assignee: nobody → Philip Roche (philroche)
status: New → Won't Fix
Changed in livecd-rootfs (Ubuntu Bionic):
status: New → Won't Fix
Changed in livecd-rootfs (Ubuntu Focal):
status: New → Won't Fix
Changed in livecd-rootfs (Ubuntu Lunar):
status: New → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 24.04.2

---------------
livecd-rootfs (24.04.2) noble; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

 -- Philip Roche <email address hidden> Thu, 26 Oct 2023 17:56:00 +0100

Changed in livecd-rootfs (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Utkarsh Gupta (utkarsh) wrote :

Please adjust the status of Noble, too.

Changed in livecd-rootfs (Ubuntu Noble):
status: Fix Released → New
Revision history for this message
Steve Langasek (vorlon) wrote :

"adjust the status of noble too"? the status showed that this change landed in noble.

Changed in livecd-rootfs (Ubuntu Noble):
status: New → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Proposed package upload rejected

An upload of livecd-rootfs to mantic-proposed has been rejected from the upload queue for the following reason: "cruft in the upload (livecd-rootfs-23.10.56/arm64-minimized-alignment.patch, .idea)".

Revision history for this message
Steve Langasek (vorlon) wrote :

An upload of livecd-rootfs to jammy-proposed has been rejected from the upload queue for the following reason: "cruft in upload (.idea*/)".

Revision history for this message
Philip Roche (philroche) wrote :

Apologies RE jammy upload. I have now re-uploaded Jammy livecd-rootfs 2.765.30 again without any '.idea' directory cruft.

Revision history for this message
Philip Roche (philroche) wrote :

Apologies RE mantic upload. I have now re-uploaded Mantic livecd-rootfs 23.10.56 again without any '.idea' directory cruft.

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello John, or anyone else affected,

Accepted livecd-rootfs into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/23.10.56 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-mantic
description: updated
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (livecd-rootfs/23.10.56)

All autopkgtests for the newly accepted livecd-rootfs (23.10.56) for mantic have finished running.
The following regressions have been reported in tests triggered by the package:

livecd-rootfs/23.10.56 (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/mantic/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Philip Roche (philroche) wrote :

I retriggered the s390x tests and they did pass https://autopkgtest.ubuntu.com/packages/l/livecd-rootfs/mantic/s390x

Revision history for this message
Timo Aaltonen (tjaalton) wrote :

missing verification

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This is missing verification, and there is a new 23.10.57 upload in mantic unapproved which does not include this change in its .changes file.

Revision history for this message
Andreas Hasenack (ahasenack) wrote : Please test proposed package

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.32 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-jammy
Revision history for this message
John Chittum (jchittum) wrote :

Mantic build verified

1. created cpc livecd-rootfs package in my personal PPA
(not public, viewable if you're in ~cloudware)
https://launchpad.net/~cloudware/+archive/ubuntu/secret-sauce-jchittum/+packages

2. launched a mantic build
(not public, but someone in ~cloudware can see: https://launchpad.net/~cloudware/+livefs/ubuntu/mantic/cpc-development/+build/543950 )

3. checked for matching `vmlinux` and not `vmlinuz`
+ [ -e chroot/boot/vmlinux ]
+ boot_filename=vmlinux

Revision history for this message
John Chittum (jchittum) wrote :

Jammy build verified

1. created cpc livecd-rootfs package in my personal PPA
(not public, viewable if you're in ~cloudware)
https://launchpad.net/~cloudware/+archive/ubuntu/secret-sauce-jchittum/+packages

2. launched a mantic build
(not public, but someone in ~cloudware can see: https://launchpad.net/~cloudware/+livefs/ubuntu/jammy/cpc-development/+build/543951 )

3. checked for matching `vmlinux` and not `vmlinuz`
+ [ -e chroot/boot/vmlinux ]
+ boot_filename=vmlinux

tags: added: verification-done verification-done-jammy verification-done-mantic
removed: verification-needed verification-needed-jammy verification-needed-mantic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 23.10.56

---------------
livecd-rootfs (23.10.56) mantic; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

 -- Philip Roche <email address hidden> Wed, 11 Oct 2023 11:41:00 +0100

Changed in livecd-rootfs (Ubuntu Mantic):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.33 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-jammy
removed: verification-done verification-done-jammy
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello John, or anyone else affected,

Accepted livecd-rootfs into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.765.34 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Philip Roche (philroche) wrote :

This bug was verified for Jammy in https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/2038957/comments/19 and the area of changes in the uploads since then does not affect this bug. As such I am marking as verification done again.

tags: added: verification-done-jammy
removed: verification-needed-jammy
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.765.34

---------------
livecd-rootfs (2.765.34) jammy; urgency=medium

  * Remove comments in the debian/control dependencies as it was causing the
    package build to fail.

livecd-rootfs (2.765.33) jammy; urgency=medium

  [ Steve Langasek ]
  * Use losetup instead of kpartx to resolve race conditions in riscv64
    image builds. LP: #2045797.

  [ Utkarsh Gupta ]
  * unminimize: Use lxd-installer to install LXD itself (LP: #2036725)

livecd-rootfs (2.765.32) jammy; urgency=medium

  [ Heinrich Schuchardt ]
  * arm: fix console parameter for ARM cloud-images (LP: #2036730)

livecd-rootfs (2.765.31) jammy; urgency=medium

  [ John Chittum ]
  * fix: add 6.5 kernel apparmor features to livecd-rootfs based on
    features of 6.5 in ubuntu/mantic. This will roll as HWE. (LP: #2037567)

livecd-rootfs (2.765.30) jammy; urgency=medium

  * Enable snap preseeding with ppc64el images where /boot/vmlinux is used
    instead of /boot/vmlinuz. (LP: #2038957)

livecd-rootfs (2.765.29) jammy; urgency=medium

  [ Steve Langasek ]
  * The chroot tmpfs mount should only be /var/lib/apt/lists, not
    /var/lib/apt; the latter breaks changes to /var/lib/apt/extended_states.
    (LP: #2036195).

livecd-rootfs (2.765.28) jammy; urgency=medium

  * Fix unminimize to correctly list packages. (LP: #1996489)
  * Install LXD snap from stable/ubuntu-<version> channel. (LP: #2036725)

livecd-rootfs (2.765.27) jammy; urgency=medium

  [ Thomas Bechtold ]
  * Do not modify /etc/ssh/sshd_config for ubuntu-cpc
    project builds (LP: #1968873)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Mon, 15 Jan 2024 16:41:18 +0100

Changed in livecd-rootfs (Ubuntu Jammy):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.