Installing linux-modules-extra-raspi in chroot triggers flash-kernel which fails

Bug #1955353 reported by guysoft
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
flash-kernel (Ubuntu)
Fix Released
Undecided
Unassigned
Jammy
Fix Released
Medium
Unassigned
linux-raspi (Ubuntu)
Invalid
Undecided
Unassigned
Jammy
Invalid
Undecided
Unassigned

Bug Description

[Impact]
If flash-kernel is installed in a chroot environment it will crash when trying to execute its postinst script. This will prevent the new design
of ubuntu-image from building any images that have flash-kernel seeded.

[Test Plan]
* Create a chroot on armhf or arm64
* sudo chroot <path_to_chroot> apt install -y flash-kernel linux-modules-extra-raspi
* Observe that flash-kernel fails due to a non-existent boot partition
* Enable -proposed in the chroot
* Attempt to install flash_kernel and linux-modules-extra-raspi again
* Observe that the packages now successfully install

[Regression Potential]
* If there are any valid existing workflows that attempt to run flash-kernel in a chroot, they could be affected by this change.
* However, there is an environment variable in place, FK_FORCE, that will enable these workflows to succeed again
* Testing should be done in some other valid non-chroot use cases to ensure no false positives in the chroot detection.

[Original Description]
Hey, following up on: https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1947601?comments=all

Its not possible install linux-modules-extra-raspi in chroot, because it activates flash-kernel which fails.
I also tried to bypass flash-kernel by moving /sbin/flash-kernel aside before installing, but then the issue of docker failing does not get fixed.

This means that a number of distros I maintain can't have docker running correctly.

Related: https://github.com/guysoft/CustomPiOS/issues/149

Output:
12:36:15 I: /boot/initrd.img is now a symlink to initrd.img-5.13.0-1011-raspi
12:36:16 Setting up linux-modules-extra-5.13.0-1011-raspi (5.13.0-1011.13) ...
12:36:20 Setting up linux-modules-5.13.0-1011-raspi (5.13.0-1011.13) ...
12:36:25 Setting up linux-modules-extra-raspi (5.13.0.1011.17) ...
12:36:25 Processing triggers for linux-image-5.13.0-1011-raspi (5.13.0-1011.13) ...
12:36:25 /etc/kernel/postinst.d/initramfs-tools:
12:36:25 update-initramfs: Generating /boot/initrd.img-5.13.0-1011-raspi
12:36:45 cryptsetup: ERROR: Couldn't find sysfs directory for 7:0
12:36:47 Label writable not found in /dev/disk/by-label
12:36:48 Warning: root device LABEL=writable does not exist
12:37:07 W: mkconf: MD subsystem is not loaded, thus I cannot scan for arrays.
12:37:07 W: mdadm: failed to auto-generate temporary mdadm.conf file.
12:38:59 flash-kernel: deferring update (trigger activated)
12:38:59 /etc/kernel/postinst.d/zz-flash-kernel:
12:38:59 flash-kernel: deferring update (trigger activated)
12:38:59 Processing triggers for flash-kernel (3.104ubuntu5) ...
12:39:00 Unsupported platform 'BCM2835'.
12:39:00 dpkg: error processing package flash-kernel (--configure):
12:39:00 installed flash-kernel package post-installation script subprocess returned error exit status 1
12:39:00 Errors were encountered while processing:
12:39:00 flash-kernel
12:39:01 needrestart is being skipped since dpkg has failed
12:39:09 E: Sub-process /usr/bin/dpkg returned an error code (1)
12:39:09 ++++ echo_red 'build failed, unmounting image...'

Things I tried:

Patching as describe here: https://forums.raspberrypi.com/viewtopic.php?t=258056
Then moving before install /sbin/flash-kernel to /sbin/flash-kernel.bak

Results in the following output:

13:14:27
13:14:27 Pending kernel upgrade!
13:14:27
13:14:27 Running kernel version:
13:14:27 5.10.81-v8+
13:14:27
13:14:27 Diagnostics:
13:14:27 The currently running kernel version is not the expected kernel version 5.13.0-1011-raspi.
13:14:27
13:14:27 Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return]
13:14:27
13:14:27 Failed to check for processor microcode upgrades.
13:14:27
13:14:27 No services need to be restarted.
13:14:27
13:14:27 No containers need to be restarted.
13:14:27
13:14:27 No user sessions are running outdated binaries.

guysoft (guysoft)
description: updated
description: updated
Revision history for this message
Juerg Haefliger (juergh) wrote :

flash-kernel should detect when it's called in a chroot environment.

Changed in linux-raspi (Ubuntu):
status: New → Invalid
Revision history for this message
guysoft (guysoft) wrote :

@juergh You wrote me in #1947601 to open a new issue, should I open a new one with a different description?

Revision history for this message
Juerg Haefliger (juergh) wrote :

No, this bug is just fine. All I'm saying is that it looks to be a flash-kernel rather than a linux-raspi issue.

Revision history for this message
Dave Jones (waveform) wrote :

Hmmm, the following line is interesting:

> 12:39:00 Unsupported platform 'BCM2835'.

That suggests that flash-kernel is falling back to querying /proc/cpuinfo (which lists the "Hardware" field as BCM2835). Ordinarily, flash-kernel relies on /proc/device-tree/model instead which suggests that maybe /proc/device-tree isn't mounted (or isn't mounted correctly) within your chroot?

You could try overriding it by writing the line "Raspberry Pi 4 Model B" to /etc/flash-kernel/machine, which takes priority over the device-tree (it doesn't have to be that specific model; pick something relevant from the "Machine:" lines in /usr/share/flash-kernel/db/all.db). However, I'm not sure what else will fail if the device-tree isn't available within the chroot (in other words, it's probably more useful to figure that out).

Revision history for this message
guysoft (guysoft) wrote :

Hey, so tested, adding Raspberry Pi 4 Model B" to /etc/flash-kernel/machine makes the dpkg install not crash.
However the modules on docker don't work until I run:

$ sudo dpkg-reconfigure flash-kernel

On the physical device after the .img has been flashed to it.

Which means it needs to flash something.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package flash-kernel - 3.106ubuntu7

---------------
flash-kernel (3.106ubuntu7) kinetic; urgency=medium

  * Change FK_FORCE_CONTAINER to FK_FORCE and cleanly exit if running
    in a chroot environment (LP: #1955353)

 -- William 'jawn-smith' Wilson <email address hidden> Thu, 08 Sep 2022 09:16:55 -0500

Changed in flash-kernel (Ubuntu):
status: New → Fix Released
Changed in linux-raspi (Ubuntu Jammy):
status: New → Invalid
Changed in flash-kernel (Ubuntu Jammy):
status: New → In Progress
importance: Undecided → Medium
tags: added: foundations-todo
description: updated
Revision history for this message
Dan Bungert (dbungert) wrote :

initramfs-tools/hooks/flash_kernel_set_root has another copy of the FK_FORCE_CONTAINER check, is that a problem? That it now has different logic for this check than what is in functions?

Revision history for this message
Dave Jones (waveform) wrote :

Yes, that should be changed to FK_FORCE (the change in the logic of FK_FORCE_CONTAINER doesn't matter in this case, but the change of variable name should be fixed). I'll upload a fix for kinetic, but this will need to be remembered for the jammy SRU.

Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello guysoft, or anyone else affected,

Accepted flash-kernel into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/flash-kernel/3.104ubuntu14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in flash-kernel (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
guysoft (guysoft) wrote :
Download full text (3.6 KiB)

Getting now: Unsupported platform ''.

+ apt-get install -y linux-modules-extra-raspi
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  linux-image-5.15.0-1017-raspi linux-modules-5.15.0-1017-raspi
  linux-modules-extra-5.15.0-1017-raspi
Suggested packages:
  fdutils linux-raspi-doc-5.15.0 | linux-raspi-source-5.15.0 linux-raspi-tools
  linux-headers-5.15.0-1017-raspi
The following NEW packages will be installed:
  linux-image-5.15.0-1017-raspi linux-modules-5.15.0-1017-raspi
  linux-modules-extra-5.15.0-1017-raspi linux-modules-extra-raspi
0 upgraded, 4 newly installed, 0 to remove and 84 not upgraded.
Need to get 66.7 MB of archives.
After this operation, 308 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 linux-modules-5.15.0-1017-raspi arm64 5.15.0-1017.19 [31.5 MB]
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 linux-image-5.15.0-1017-raspi arm64 5.15.0-1017.19 [10.5 MB]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 linux-modules-extra-5.15.0-1017-raspi arm64 5.15.0-1017.19 [24.7 MB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 linux-modules-extra-raspi arm64 5.15.0.1017.16 [2334 B]
Fetched 66.7 MB in 3s (19.4 MB/s)
                                 Selecting previously unselected package linux-modules-5.15.0-1017-raspi.
(Reading database ... 69605 files and directories currently installed.)
Preparing to unpack .../linux-modules-5.15.0-1017-raspi_5.15.0-1017.19_arm64.deb ...
Unpacking linux-modules-5.15.0-1017-raspi (5.15.0-1017.19) ...
Selecting previously unselected package linux-image-5.15.0-1017-raspi.
Preparing to unpack .../linux-image-5.15.0-1017-raspi_5.15.0-1017.19_arm64.deb ...
Unpacking linux-image-5.15.0-1017-raspi (5.15.0-1017.19) ...
Selecting previously unselected package linux-modules-extra-5.15.0-1017-raspi.
Preparing to unpack .../linux-modules-extra-5.15.0-1017-raspi_5.15.0-1017.19_arm64.deb ...
Unpacking linux-modules-extra-5.15.0-1017-raspi (5.15.0-1017.19) ...
Selecting previously unselected package linux-modules-extra-raspi.
Preparing to unpack .../linux-modules-extra-raspi_5.15.0.1017.16_arm64.deb ...
Unpacking linux-modules-extra-raspi (5.15.0.1017.16) ...
Setting up linux-image-5.15.0-1017-raspi (5.15.0-1017.19) ...
I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-1017-raspi
I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-1017-raspi
Setting up linux-modules-5.15.0-1017-raspi (5.15.0-1017.19) ...
Setting up linux-modules-extra-5.15.0-1017-raspi (5.15.0-1017.19) ...
Setting up linux-modules-extra-raspi (5.15.0.1017.16) ...
Processing triggers for linux-image-5.15.0-1017-raspi (5.15.0-1017.19) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-5.15.0-1017-raspi
cryptsetup: ERROR: Couldn't find sysfs directory for 7:31
cryptsetup: ERROR: Couldn't find sysfs directory for 259:7
Label writable not found in /dev/disk/by-label
Warning: root device LABEL=writable does not exist
W: Couldn't identify type of root file system for fsck hook
flash-kernel: deferrin...

Read more...

Revision history for this message
Dave Jones (waveform) wrote :

Are you sure you've got the -proposed version of flash-kernel installed? Currently the fix is only in -proposed pending migration after validation. The "Unsupported platform" check is well after the FK_FORCE check which would suggest you're still running the current version of flash-kernel in jammy-updates (3.104ubuntu13) rather than the -proposed version (3.104ubuntu14).

You can find information on enabling the -proposed pocket at https://wiki.ubuntu.com/Testing/EnableProposed. In the meantime I'll try and get around to validating this myself this week, but things are a bit busy with the upcoming kinetic release currently.

Revision history for this message
guysoft (guysoft) wrote (last edit ):

Ok indeed it was actually using 3.104ubuntu12. Even though I did add the proposed repo.

I was installing linux-modules-5.15.0-1017-raspi which was triggering flash-kernel.
So I had before installing that to run explicitly:
apt-get install -y flash-kernel.

Looks like it does not break now, I will have to flash the built image on to a PI and test if all is working too. Hope to have a new nightly build of UbuntuDockerPi soon so anyone can just flash and test.

Revision history for this message
William Wilson (jawn-smith) wrote :

The verification passed for jammy:

```
ubuntu@ubuntu:~$ sudo chroot workjawn/chroot/ apt-cache policy flash-kernel
flash-kernel:
  Installed: (none)
  Candidate: 3.104ubuntu13
  Version table:
     3.104ubuntu13 500
        500 http://ports.ubuntu.com jammy-updates/main arm64 Packages
     3.104ubuntu9 500
        500 http://ports.ubuntu.com jammy/main arm64 Packages
ubuntu@ubuntu:~$ sudo chroot workjawn/chroot/ apt install flash-kernel linux-modules-extra-raspi
<truncated irrelevant output>
Errors were encountered while processing:
 flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)
ubuntu@ubuntu:~$ sudo chroot workjawn/chroot/ apt-cache policy flash-kernel
flash-kernel:
  Installed: 3.104ubuntu13
  Candidate: 3.104ubuntu14
  Version table:
     3.104ubuntu14 500
        500 http://ports.ubuntu.com jammy-proposed/main arm64 Packages
 *** 3.104ubuntu13 500
        500 http://ports.ubuntu.com jammy-updates/main arm64 Packages
        100 /var/lib/dpkg/status
     3.104ubuntu9 500
        500 http://ports.ubuntu.com jammy/main arm64 Packages
ubuntu@ubuntu:~$ sudo chroot workjawn/chroot/ apt install flash-kernel linux-modules-extra-raspi
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
linux-modules-extra-raspi is already the newest version (5.15.0.1017.16).
flash-kernel is already the newest version (3.104ubuntu14).
0 upgraded, 0 newly installed, 0 to remove and 32 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
E: Can not write log (Is /dev/pts mounted?) - posix_openpt (19: No such device)
Setting up initramfs-tools (0.140ubuntu13) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for flash-kernel (3.104ubuntu14) ...
Processing triggers for initramfs-tools (0.140ubuntu13) ...
update-initramfs: Generating /boot/initrd.img-5.15.0-1017-raspi
Warning: root device does not exist
ubuntu@ubuntu:~$ sudo chroot workjawn/chroot/ echo $?
0
```

tags: added: verification-done verification-done-jammy
removed: foundations-todo verification-needed verification-needed-jammy
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package flash-kernel - 3.104ubuntu14

---------------
flash-kernel (3.104ubuntu14) jammy; urgency=medium

  * Change FK_FORCE_CONTAINER to FK_FORCE and cleanly exit if running
    in a chroot environment (LP: #1955353)

 -- William 'jawn-smith' Wilson <email address hidden> Fri, 07 Oct 2022 09:14:11 -0500

Changed in flash-kernel (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for flash-kernel has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.