3A+ boot failure on Eoan

[Dave Jones]

The current Eoan image hits a kernel panic when booting on a Raspberry Pi 3A+. Unfortunately it does so before the serial console has been enabled so there's no useful output beyond what scrolls by rapidly on the framebuffer console.

It is notable that the start.elf bootloader (prior to u-boot), loads the device-tree for the 3B+ because no device-tree specific to the 3A+ exists (in the 27*.dtb set; there is one in the 28*.dtb set but I'm not convinced those are used by anything); u-boot proceeds successfully but then the kernel panics.

This occurs on both the armhf and arm64 architectures.

[Dave Jones (waveform)]

Related to #1847596 - commenting out the vc4-fkms-v3d overlay in config.txt permits the 3A+ to boot, albeit with corrupted framebuffer. In other words, solving #1847596 "properly" should also yield a fix for this.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1848247

[Hui Wang (hui.wang)]

So far, what I got from investigation is:

After setting vc4-fkms-v3d, the fb driver's sequence is bootloader -> simplefb -> drm_vc4 fb

Without vc4-fkms-v3d, the sequence is bootloader -> simplefb -> bcm2708_fb (the screen will flicker when bcm2708_fb takeover the display)

I will continue looking at the bcm2708_fb.

And for 3A+ booting issue, after adding vc4-fkms-v3d, the system need much memory, but 3A+ only have 512M physical memory, looks like the 512M is not enough for drm_vc4, so the kernel crash with "out of memory errors".

[Dave Jones (waveform)]

I'm at a similar place; currently digging into the simplefb side of things. My current thoughts are as follows:

1. Ideally I want to ditch the vc4-fkms-v3d overlay; firstly this resolves the 3A+ booting issue, secondly it's something that should be an option rather than mandatory to the boot procedure

2. It appears the vc4-fkms-v3d overlay also prevents u-boot from operating the framebuffer correctly. Which is just another reason to get rid of it :)

3. I note that the simplefb is only set up during the bootm phase, i.e. at the *end* of u-boot's run. However, long before that (without the overlay in place) u-boot is happily displaying things on the framebuffer. Which makes me wonder if the simplefb is needed at all; bcm2708_fb is obviously capable of generating its own framebuffer (given the kernel can be launched without u-boot).

4. In the device-tree generated by the simplefb code I note that the physical dimensions of the framebuffer it creates include overscan margins; when the overlay is in place I don't see any overscan margins. Incorrect dimensions (or strides) is something that could well produce the sort of corruption we're seeing.

Anyway, my next step is going to be a crude excision of the simplefb code from u-boot (or the code that calls it) to see if that fixes things. If it does it'll still be a bit of a hack, but a better hack than the overlay. Obviously I'd ultimately like to fix the framebuffer code "properly" (not least, including on the Pi4 where u-boot's framebuffer doesn't operate at all) but I'll take a better hack that enables 3A+ to boot in the meantime.

[Hui Wang (hui.wang)]

I mean the simplefb driver in the kernel in the #3, I didn't know there is a simplefb driver in the uboot too.

So it looks like: uboot.simplefb ---> kernel.simplefb works fine
                  uboot.simplefb ---> kernel.simplefb ----> drm_vc4_fb works fine (but RPI3A+ crash)
                  uboot.simplefb ---> kernel.simplefb ---> kernel.bcm2708_fb makes the screen flicker

[Dave Jones (waveform)]

> I mean the simplefb driver in the kernel in the #3, I didn't know there is a simplefb driver in the uboot too.

Ah, I'm afraid my lack of kernel knowledge is showing there: I didn't realize the was a simplefb in the kernel! I should've thought given the kernel starts off looking fine, and only shortly after starting de-syncs the framebuffer (presumably the point where bcm2708_fb takes over).

Confirmed last night that the u-boot simplefb is largely doing the right thing, although I did run across some bugs/bad assumptions in the framebuffer mailbox handling, but nothing that should affect anything here (it doesn't check the actual result of setting depth / alpha-mode / pixel-order; I'll put together a patch for that later).

[Dave Jones (waveform)]

Can confirm that the test kernel from LP: #1848790 comment #11 fixes boot on 3A+ (remove vc4-fkms-v3d overlay from config.txt too, and boot operates correctly with framebuffer working). Tested on arm64 3A+ (and 3B, 3B+, 4B); need to test on armhf too.

[Brian Murray (brian-murray)]

The Eoan Ermine has reached end of life, so this bug will not be fixed for that release