flash-kernel only updates the dtb of the Pi it's on

[Dave Jones]

[Impact]

When flash-kernel is executed to copy things to the boot partition, only the dtb of the Pi that it is being executed on is copied. In order to support moving the SD card between Pi models, flash-kernel needs to copy *all* dtbs provided by the kernel to the boot partition.

Consider the case of SRUing Pi 4 support to Bionic. Bionic does not currently support the Pi 4, so a user must necessarily boot their Bionic image on, say, a Pi 3. If they then upgrade to a kernel supporting the Pi 4, flash-kernel will only copy the dtb for the Pi 3 to the boot partition. If the card is then moved to a Pi 4 it will fail to boot due to a missing device-tree.

This would be a very welcome bugfix/feature for eoan as well, since that's where we had first, proper pi4 support.

[Test Case]

On both armhf and arm64:

1. Flash 19.10 image to an SD card
2. Boot the card on a Pi 2B (armhf only), 3B, or 3B+
3. Upgrade the kernel to Hui's test kernel, fixing the 4B 4Gb USB issue (see https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1848790/comments/11); this comes with a new device-tree for the 4B - the device-tree for the 4B shipped with the image will not work with the new kernel
4. Shut down the machine
5. Move the card to a Pi 4B (any RAM amount)
6. Attempt to boot the machine; observe failure (the linux kernel will start, but boot will not complete and serial will not operate)
7. Check the content of the boot partition; only one DTB (the original Pi's) should have a ".bak" copy

In order to test that the fix works repeat the procedure, but before step 3, upgrade flash-kernel from the PPA mentioned below (https://launchpad.net/~waveform/+archive/ubuntu/flash-kernel/). Now in step 6, the boot should work, and in step 7, all dtbs (and *.dtbo in the overlays dir) should have ".bak" copies.

[Regression Potential]

The fix comes with a fair amount of flash-kernel code cleanup. On one side, more changes might feel more regression prone, but on the other - since we are doing a direct backport from focal, this means that there's no chance of us forgetting a change and making the backport incomplete.

There is risk involved, as this SRU changes how basically flash-kernel performs updates for all the pi variants. The test suite however has been extended, so as to reduce regression potential. Potential regressions should be looked for in missing DTBs during flash-kernel updates and broken boot (due to the modifications made to the boot script). That being said, most of those issues should be instantly visible during verification.

[Dave Jones (waveform)]

Fixed version of flash-kernel available from ppa:waveform/flash-kernel at:

    https://launchpad.net/~waveform/+archive/ubuntu/flash-kernel/

Built from branch at (final commit is the version bump; remove this for proposed commits):

    https://code.launchpad.net/~waveform/ubuntu/+source/flash-kernel/+git/flash-kernel/+ref/pi4-upgrade

The proposed fix also incorporates the changes required for LP: #1841955.

[Launchpad Janitor (janitor)]

This bug was fixed in the package flash-kernel - 3.98ubuntu6

---------------
flash-kernel (3.98ubuntu6) focal; urgency=medium

  * Remove redundant DTB-Kernel-Version field
  * Remove redundant copying of boot dtb
  * Fix mkimage_kernel tests
  * Add "pi" value for the "Method" field which copies all dtbs and overlays
    to the boot partition and provides defaults for the boot paths of the
    kernel, initrd, and u-boot script. This fixes upgrades to support the Pi 4
    on Bionic (LP: #1850678)
  * Update pi bootscript to support all flash-kernel vars, including
    calculated devtype and partition for future USB boot support
  * Add entry for Raspberry Pi 3A+ (LP: #1841955)

 -- Dave Jones <email address hidden> Thu, 07 Nov 2019 14:02:26 +0000

[Brian Murray (brian-murray)]

Hello Dave, or anyone else affected,

Accepted flash-kernel into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/flash-kernel/3.98ubuntu5.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Dave Jones (waveform)]

Tested on 3, 3+, CM3, CM3+, and 4 under armhf and arm64 with eoan-proposed image. No regressions detected.

[Launchpad Janitor (janitor)]

This bug was fixed in the package flash-kernel - 3.98ubuntu5.1

---------------
flash-kernel (3.98ubuntu5.1) eoan; urgency=medium

  * Backport changes from 3.98ubuntu6 to eoan
  * Remove redundant DTB-Kernel-Version field
  * Remove redundant copying of boot dtb
  * Fix mkimage_kernel tests
  * Add "pi" value for the "Method" field which copies all dtbs and overlays
    to the boot partition and provides defaults for the boot paths of the
    kernel, initrd, and u-boot script. This fixes upgrades to support the Pi 4
    on Bionic (LP: #1850678)
  * Update pi bootscript to support all flash-kernel vars, including
    calculated devtype and partition for future USB boot support
  * Add entry for Raspberry Pi 3A+ (LP: #1841955)

 -- Dave Jones <email address hidden> Thu, 07 Nov 2019 14:02:26 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for flash-kernel has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package flash-kernel - 3.98ubuntu11~18.04.1

---------------
flash-kernel (3.98ubuntu11~18.04.1) bionic; urgency=medium

  * Permit flash-kernel to workaround an inconsistent /boot directory when
    triggered prior to the kernel finishing its triggers (LP: #1861292)

flash-kernel (3.98ubuntu10~18.04.1) bionic; urgency=medium

  * SRU of 3.98ubuntu10 to support Raspberry Pi 4B. (LP: #1847587)
  * Removed new entries from db/all.db for machines that were not already
    supported on Bionic
  * Remove "quiet splash" from kernel command line options; Bionic shipped
    without these options and we shouldn't change the boot experience for
    existing users
  * Ensure the "overlays" dir always exists on the boot partition; the Bionic
    image for the Pi 2 shipped without this dir

 -- Dave Jones <email address hidden> Wed, 29 Jan 2020 16:15:05 +0000