Create an image for all raspberry models

[Adam Smith]

Currently (Xenial to Bionic) server images only run on the raspberry pi 2. This is due to the u-boot bootloader.

However it is possible to produce one image that runs on a pi 2 and 3. Fedora arm 32 bit produce such an image. They make use of conditional filters in the config.txt (https://www.raspberrypi.org/documentation/configuration/config-txt/conditional.md ) and put two versions of u-boot on the image.

They other way to do it is to remove u-boot entirely and just rely on the internal pi bootloader to sort it out. This could be permanent, or just for first boot where upon flash-installer is called to install the correct u-boot file.

[Adam Smith (adamsmith)]

Trying the latest opensuse armhf image, they seem to have a U-boot.bin that works on the pi 2 and 3.

[Oliver Grawert (ogra)]

note that we have the ability to build unified UbuntuCore images, but to my knowledge nobody works on support for this in classic ubuntu images (server/desktop) ...

the source for the Core gadget is at

https://github.com/ogra1/pi-kiosk-gadget

more details are at:

https://forum.snapcraft.io/t/rfc-single-universal-raspberry-pi-gadget-and-image/6634

a demo appliance image (for a digital signage demo) using this setup can be found at:

http://people.canonical.com/~ogra/snappy/kiosk/dashkiosk-server.img.xz

[Adam Smith (adamsmith)]

Oliver, thanks for the comment. Interesting that core has started to implement this. I don't know much about snaps/gadgets.

The rpi2 u-boot is supposed to work with the pi3 if the pi3-miniuart-bt overlay is used. However, the single u-boot file opensuse uses appears to be doing something clever. It is either using and acting on the device tree that is passed to it by the firmware, or it is detecting if I have a serial cable connected.

The way Ubuntu classic boots the pi could do with an overhaul. It is overcomplicated for no benefit that I can see. I'm willing to put a bit of work into the classic images if there is a consensus on what should be done. It seems pointless blindly coming up with patches at the moment.

I think flash-kernel should be dropped since it breaks on every board revision and it can't handle overlays. It wouldn't be hard to come up with some simple kernel hooks and put them in linux-firmware-raspi2. This is what Debian's raspi3-firmware package should of done, but they went crazy over the top.

u-boot as implemented at the moment brings no benefit over the pi's built in firmware bootloader. u-boot should load the latest kernel from /boot or provide a menu or something useful. If it does none of these things then it should be got rid of.

My preferred long term goal would be for Ubuntu to adopt grub2 on arm. This is what opensuse uses (via u-boot acting as uefi). As far as I can tell grub 2 cannot handle overlays at the moment. U-boot can load overlays, but Fedora, opensuse and Ubuntu all use the pi's firmware to apply overlays. This could continue, with linux-firmware-raspi2 updating the dtbs/overlays files on /boot/efi or /boot/firmware when appropriate (e.g. an update from 18.10 to 19.04).

[Oliver Grawert (ogra)]

flash-kernel is the thing that puts the bootloader and bootloader configuration in place at the end of installations (the equivalent to grub-install on x86).
It is the interface/API you need if you want to use ubiquity or debian-intaller.
It is also hooked up directly with the kernel debs and automatically run after a kernel update (to adjust bootloader variables, write things to flash on systems where thats needed etc).

while you can indeed go completely without debian-installer/ubiquity and use self-expanding pre-installed images, you would need to special-case a bunch of expected distro-defaults in the images... and would have to maintain this patchset.

my personal preference would also be to chainload grub from u-boot like opensuse and fedora do to implement UEFI, but this requires a lot of integration work in various places of the distro. this should be done as a general solution for all armhf installs across the board and needs a bit more planning than just throwing together a rpi image ... that said, as a proof of concept, a pi image would indeed be a good start.

note though that in the end you will still need some generic api/interface for the installers if you want to use them. which will make you end up with a flash-kernel-like tool again.

[Adam Smith (adamsmith)]

I understand why flash-kernel was used, and in the past I myself have suggested its use (https://bugs.launchpad.net/raspbian/+bug/1723203 ). I've created installers for the pi using ubiquity (https://ubuntu-mate.community/t/aarch64-on-raspberry-pi-2-rev-1-2-3b-3b/16853 ) and the debian-installer, which is why I'm convinced flash-kernel is not fit for purpose on the pi.

To my knowledge there are currently no installers that use flash-kernel as intended on the pi. Currently you can't boot the generic armhf kernel on the pi (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1795869 ), and the generic kernel uses different dtb names, so flash-kernel doesn't work even if you could boot the debian-installer. The Linux-raspi2 kernel doesn't have any udebs, nor does the debian-installer recognise the kernel (it absolutely refuses to install it on the target system).

If arm is switching to grub2, then grub-installer will be run by ubiquity/debian-installer instead of flash-kernel-installer.

[Adam Smith (adamsmith)]

The current thinking seems to be that efi passes grub2 the flattened device tree. The problem with this is that the dtb is not at all stable. It is supposed to be independent of operating system, but there are differences between the upstream and rpi foundation kernels. Grub2 or U-boot can give the user the choice of kernel or even operating system, but it could be fed an incompatible dtb by efi. Grub2 can load a device tree which solves this, but you currently loose the overlays.

Having said all of that, the dtb is stable enough not to require always updating on every kernel update. This is how flash-kernel can get away with not updating the overlays. But the safest thing is not to update the dtb, so that the overlays are in-sync. Kernel drivers are supposed to (if they are written correctly) be able to cope with an old dtb file.

I suggest that Linux-firmware-raspi2 handles updating the dtb files, and it does so infrequently and only when necessary.

[Adam Smith (adamsmith)]

Note it is currently not possible to boot the pi in Ubuntu armhf using uefi. As previously mentioned, the generic kernel is missing pi modules, and Linux-raspi2 is missing efi-stub https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1710517

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dave Jones (waveform)]

Going back through old bugs - I think we can mark this "Fix Released" at this point; the classic and core images are now unified (while the +raspi2, +raspi3, +cm3 images *appear* to exist in some cases these are now just links to the unified image).

Just to address a couple of the other comments above:

* I updated flash-kernel last year to add a special-cased "pi" method that now copies *all* discovered DTBs (and overlays) when it runs, permitting an SD card to be moved between support Pis (this was necessary for the upgrade story on Bionic to support the Pi4 on existing installations).

* u-boot is a bit of a pain on the images but we do use it for two relatively important things:

* on Core it's used to support the A/B boot procedure. This requires a bootloader with writeable state (the Pi's built-in bootloader is deliberately read-only for simplicity and safety, but this means it cannot support an A/B boot case).

* on Classic (and Core in future) it's used to support decompressing the arm64 kernel which doesn't support self-extracting compression.

So, for the time being at least, u-boot and flash-kernel are likely to stay. As far as future designs are concerned, I don't particularly like the EFI option as it means adding yet *another* component into our boot process (alongside u-boot). I'd much rather move the opposite way and remove a component (u-boot), just relying on the pi's bootloader. However, that would require the pi's bootloader to grow features covering the two cases above (which isn't out of the question, but I've no idea if/when that may happen).

As for flash-kernel - I don't see that going away at the moment (for the reasons ogra's mentioned above), and with the "pi" special case in there I'm reasonably happy with it staying.