Recent updates broke Ubuntu on Raspberry Pi 3

[zubozrout]

Hello,
I've been running Raspberry Pi 3 with Ubuntu as described here https://wiki.ubuntu.com/ARM/RaspberryPi

I've installed Ubuntu 16.04 and then upgraded to 16.10 and have been using this setup successfully since October up until last week. Unfortunately one of the latest kernel updates broke the installation and I found out the same happened for Ubuntu 16.04 LTS - and therefore I can no longer run up-to-date Ubuntu installation (with updated kernel) on my RPI.

I am getting this error:
Error image is not a fdt - must reset the board to recover

I also tried this but it didn't help me: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=168838

Now I know running Ubuntu on RPI3 this way is not officially supported, but since Ubuntu Snappy is supported I thought someone here could know what is going on.

Thank you very much.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1652504

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Markus Birth (mbirth)]

Same here. The update to 1021 breaks the boot and the only way to recover is to rename all *.bak files back so it boots the older kernel.

[Markus Birth (mbirth)]

The actual problem happens earlier in the process than depicted in the screenshot above:

    Found U-Boot script /boot.scr
    reading /boot.scr
    412 bytes read in 13 ms (30.3 KiB/s)
    ## Executing script at 02000000
    libfdt fdt_check_header(): FDT_ERR_BADMAGIC
    No FDT memory address configured. Please configure
    the FDT address via "fdt addr <address>" command.
    Aborting!

[Markus Birth (mbirth)]

It seems the FDT isn't loaded properly. I can boot the system if I load the FDT manually and do the remaining steps from the boot.scr manually:

    fatload mmc 0:1 0x02000000 bcm2710-rpi-3-b.dtb
    fdt addr 0x02000000
    fatload mmc 0:1 0x01000000 vmlinuz
    fatload mmc 0:1 0x02100000 initrd.img
    setenv initrdsize $filesize
    bootz 0x01000000 0x02100000:${initrdsize} 0x02000000

[Markus Birth (mbirth)]

Downgrading the kernel or recently upgraded packages flash-kernel or linux-firmware-raspi2 didn't work for me. But you can fix this temporarily by editing /usr/share/flash-kernel/bootscript/bootscr.rpi3 and adding the line

    fatload mmc 0:1 ${fdt_addr_r} bcm2710-rpi-3-b.dtb

after the line

    setenv fdt_addr_r 0x02000000

Then run:

    update-initramfs -u -k all

To update the boot.scr in /boot/firmware/

[Markus Birth (mbirth)]

Upon further inspection: The flash-kernel from the rpi3 PPA https://launchpad.net/~ubuntu-raspi2/+archive/ubuntu/ppa-rpi3 is only published for Xenial and there, the boot script sets the fdt address to 0x100 instead of 0x2000000. And that works without explicitly loading the fdt.

So it seems I had the "old" flash-kernel from before upgrading to Yakkety and that worked. The new version tries to load the fdt from 0x2000000 where it isn't located.

[Markus Birth (mbirth)]

Also the official flash-kernel_3.0~rc.4ubuntu64_armhf.deb sets the fdt address to 0x100 whereas 3.0~rc.4ubuntu64.1.1 from yakkety-updates sets it to 0x02000000. So it seems to be a regression in that package. Downgrading to 3.0~rc.4ubuntu64 and running "sudo flash-kernel" works. (Don't know why it didn't work when I tried that a few hours ago, probably forgot to run the command to update the boot.scr.)

[Oliver Grawert (ogra)]

the newer firmware (start.elf,fixup.dat and bootcode.bin) hard-requires 0x02000000 because it initializes the hardware differently in the newer version...

looks like linux-firmware-raspi2 was not properly updated alongside with flash-kernel and the kernel itself, there is a hard lockstep between these three packages, it will only work if all of them are upgraded together ...

perhaps the linux-raspi2 metapackage should introduce a versioned dependency for the two others to avoid such fallout in the future ...

[Markus Birth (mbirth)]

Could the missing fdt at 0x02000000 be the reason, the RasPi now has a different MAC address on each and every boot? Because I noticed this after I got the boot process working again.

[Oliver Grawert (ogra)]

@markus: not sure, i dont know how the non snappy images set the MAC, technically if they use similar uboot scripts the MAC should come from the first dtb snippet that is read from the ROM/closed bootloader, that part should not be related to the actual dtb address at all...

[Paolo Pisati (p-pisati)]

What's the content of /boot/firmware/config.txt?

[Markus Birth (mbirth)]

I don't have access to the RasPi3 at the moment (due to the changing MAC screwing with DHCP and port forwardings). I'll see what I can do.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Florian Friedrich (ffried)]

Same for me. Using Ubuntu 16.04.1 LTS on a RasPi3.
Adding the line

    fatload mmc 0:1 ${fdt_addr_r} bcm2710-rpi-3-b.dtb

to /usr/share/flash-kernel/bootscript/bootscr.rpi3 helped me, too. Thanks @mbirth!
Also, I have the same issue with the MAC address. I'm currently using macchanger to set it back to what it should be on every launch.

This is the content of my /boot/firmware/config.txt:

# For more options and information see
 # http://www.raspberrypi.org/documentation/configuration/config-txt.md
 # Some settings may impact device functionality. See link above for details

 kernel=uboot.bin

 # enable uart (needed for u-boot on RPi3)
 enable_uart=1

 # set extended DT area
 device_tree_address=0x100
 device_tree_end=0x8000

 # enable i2c
 dtparam=i2c_arm=on
 dtparam=spi=on

 # uncomment if you get no picture on HDMI for a default "safe" mode
 #hdmi_safe=1

 # uncomment this if your display has a black border of unused pixels visible
 # and your display can output without overscan
 #disable_overscan=1

 # uncomment the following to adjust overscan. Use positive numbers if console
 # goes off screen, and negative if there is too much border
 #overscan_left=16
 #overscan_right=16
 #overscan_top=16
 #overscan_bottom=16

 # uncomment to force a console size. By default it will be display's size minus
 # overscan.
 #framebuffer_width=1280
 #framebuffer_height=720

 # uncomment if hdmi display is not detected and composite is being output
 #hdmi_force_hotplug=1

 # uncomment to force a specific HDMI mode (this will force VGA)
 #hdmi_group=1
 #hdmi_mode=1

 # uncomment to force a HDMI mode rather than DVI. This can make audio work in
 # DMT (computer monitor) modes
 #hdmi_drive=2

 # uncomment to increase signal to HDMI, if you have interference, blanking, or
 # no display
 #config_hdmi_boost=4

 # uncomment for composite PAL
 #sdtv_mode=2

 #uncomment to overclock the arm. 700 MHz is the default.
 #arm_freq=800

I guess the following part is the problem?

 # set extended DT area
 device_tree_address=0x100
 device_tree_end=0x8000

[Markus Birth (mbirth)]

I just got access to that Raspberry Pi 3 again. I saw that the expected fdt address now changed to 0x02008000 in the boot script.

In the config.txt, it was set as:

# set extended DT area
device_tree_address=0x100
device_tree_end=0x8000

So I changed it to:

# set extended DT area
device_tree_address=0x2008000
device_tree_end=0x200ff00

And voilà - it works. Even the MAC address is correct now.

[Florian Friedrich (ffried)]

Yes! By also setting the `device_tree_end`, I'm getting the correct MAC address now, too. Thanks again @mbirth!

[Avio (aviopene)]

Workaround in comment #16 worked also for me, thank you.

[Felipe Reyes (freyes)]

workaround in #16 worked. Thanks @mbirth

[ted (teddy7x)]

I spent most of the day last weekend re-writing my sdcard over and over again trying to come up with some magic apt-get upgrade combination that didn't brick the pi. I ended up accidentally overwriting a card I have stuff I care about on. I want to personally thank whoever added the code that broke boot after upgrade. Post #16 did work for me, but whoever wrote the code that broke it deserves a gold star. you sir/madam are a winner

[NB (nboutelier)]

Has anyone else noticed that the #16 fix to this breaks the bcm2835-v4l2 driver? modprobe bcm2835-v4l2 crashes the system.

[JoergH (joergh)]

I don't know about the breakage, but I was just bitten by this bug after an upgrade, and #16 fixed the issue for me, too.

[ajmal (ajmalhinas-gmail)]

 modprobe bcm2835-v4l2 crashes the system. I tried various thinks but same. I could not use openCV to capture the images. However raspicam_node (https://github.com/fpasteau/raspicam_node) able is capturing images from the camera.

HARDWARE: Raspberry Pi3
Camera: picam v2
uname -vr output: 4.4.0-1080-raspi2 #88-Ubuntu SMP Mon Dec 11 14:23:15 UTC 2017

[Chris E (cbz)]

As of upgrading between 16.04 and 18.04.2 the fdt_addr has changed again, I was able to fix this via the following, first cat boot.scr on the primary partition, there's a bunch of binary at the start of the file followed by fdt_addr_r 0x03000000

Use that as the base value on config.txt:

device_tree_address=0x03000000

You do not need to set device_tree_end, this got the pi booting again.