Overlay files don't get applied

[Paolo Pisati]

Impact:

While working on a bug that involved loading an overlay dtb files upon boot, i noticed that, no matter what i put in config.txt::dtoverlay=..., no overlay was loaded at all.

After further investigation i found that recently (~April 2017), the raspberry BSP changed the overlay files suffix from -overlay.dtb to .dtbo, and while the upstream bootloader was updated to accept the new scheme, we keep shipping an older bootloader in the the archive that didn't know about this change.

To make this bug more annoying, the bootloader doesn't emit an error or a warning if it fails to load an overlay, it silently ignores it and keep going.

Ubuntu core #stable isn't affected by this bug since it's shipping it's own overlay files in a static tarball (using the old scheme), and #edge (that is where new daily images are built) is shipping an older kernel with the old scheme.

While #beta and #candidates should be affected.

Fix:

A way to fix this bug would be to update the raspi2 bootloader to a more recent version, but since updating it has proven challenging in the past (multiple images from multiple sources, #updates vs #security, different boot configs, etc), i'm taking a more straightforward approach by reverting the dtb overlay files rename back to the previous scheme.

With that said, i'm concurrently opening another LP bug (1703799) because we must update the bootloader and revert this fix as soon as the new bootloader lands in the archive.

How to test:

The are different ways you can test this (like modifying the config.txt::dtoverlay=... parameter and then checking that the corresponding hardware started to work, but then you would never know if it's a kernel driver problem, a dtb merge problem, wrong wiring, failing hardware, etc), but the best way is to use the dtc to dump out the dts structure before and after the dtoverlay
modification and then diff the two:

vanilla image:

$ dtc -I fs -O dts /proc/device-tree > pi3-vanilla.dts

now install the new kernel with the revert (dpkg -i ... or snap install ...), modify dtoverlay, reboot and dump again:

# echo "dtoverlay=vc4-kms-v3d" >> /boot/firmware/config.txt
# reboot

$ dtc -I fs -O dts /proc/device-tree > pi3-vc4.dts

check if the dtb passed to the kernel has changed:

$ diff -du pi3-vanilla.dts pi3-vc4.dts
[check that there's actually a diff and something sensible was applied]

if you are doing this on ubuntu-core, config.txt is located in /boot/uboot/config.txt, while if you are doing this in ubuntu classic, remember to manually copy the overlay files that are shipped with the new kernel:

# rm -rf /boot/firmware/overlays
# cp /lib/firmware/`uname -r`/device-tree/overlays /boot/firmware

--

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-raspi2 - 4.4.0-1067.75

---------------
linux-raspi2 (4.4.0-1067.75) xenial; urgency=low

  [ Ubuntu: 4.4.0-89.112 ]

  * CVE-2017-7533
    - dentry name snapshots

linux-raspi2 (4.4.0-1066.74) xenial; urgency=low

  * linux-raspi2: 4.4.0-1066.74 -proposed tracker (LP: #1705273)

  * Overlay files don't get applied (LP: #1703802)
    - Revert "dts, kbuild: dtbs_install installs .dtbo files too"
    - [SAUCE] Revert "BCM270X_DT: Add a .dtbo target, use for overlays" and
      subsequent

  [ Ubuntu: 4.4.0-88.111 ]

  * linux: 4.4.0-88.111 -proposed tracker (LP: #1705270)
  * [Xenial] nvme: Quirks for PM1725 controllers (LP: #1704435)
    - nvme: Quirks for PM1725 controllers
  * Upgrade Redpine WLAN/BT driver to ver. 1.2 (production release)
    (LP: #1697829)
    - SAUCE: Redpine: Upgrade to ver. 1.2 production release
  * ubuntu/rsi driver has several issues as picked up by static analysis
    (LP: #1694733)
    - SAUCE: Redpine: Upgrade to ver. 1.2 production release
  * Redpine vendor driver - Switching to AP mode causes kernel panic
    (LP: #1700941)
    - SAUCE: Redpine: Upgrade to ver. 1.2 production release
  * CVE-2017-10810
    - drm/virtio: don't leak bo on drm_gem_object_init failure
  * Ath10k to read different board data file if specify in SMBIOS (LP: #1666742)
    - ath10k: search SMBIOS for OEM board file extension
  * make snap-pkg support (LP: #1700747)
    - SAUCE: make snap-pkg support
  * ISST-LTE: Briggs:Stratton:UbuntuKVM: ics_opal_set_affinity on host kernel
    log using Intel X710 (i40e driver) (LP: #1703663)
    - i40e: use valid online CPU on q_vector initialization
  * Update snapcraft.yaml (LP: #1700480)
    - snapcraft.yaml: various improvements
  * Xenial update to 4.4.76 stable release (LP: #1702863)
    - ipv6: release dst on error in ip6_dst_lookup_tail
    - net: don't call strlen on non-terminated string in dev_set_alias()
    - decnet: dn_rtmsg: Improve input length sanitization in
      dnrmg_receive_user_skb
    - net: Zero ifla_vf_info in rtnl_fill_vfinfo()
    - af_unix: Add sockaddr length checks before accessing sa_family in bind and
      connect handlers
    - Fix an intermittent pr_emerg warning about lo becoming free.
    - net: caif: Fix a sleep-in-atomic bug in cfpkt_create_pfx
    - igmp: acquire pmc lock for ip_mc_clear_src()
    - igmp: add a missing spin_lock_init()
    - ipv6: fix calling in6_ifa_hold incorrectly for dad work
    - net/mlx5: Wait for FW readiness before initializing command interface
    - decnet: always not take dst->__refcnt when inserting dst into hash table
    - net: 8021q: Fix one possible panic caused by BUG_ON in free_netdev
    - sfc: provide dummy definitions of vswitch functions
    - ipv6: Do not leak throw route references
    - rtnetlink: add IFLA_GROUP to ifla_policy
    - netfilter: xt_TCPMSS: add more sanity tests on tcph->doff
    - netfilter: synproxy: fix conntrackd interaction
    - NFSv4: fix a reference leak caused WARNING messages
    - drm/ast: Handle configuration without P2A bridge
    - mm, swap_cgroup: reschedule when neeed in swap_cgroup_swapoff()
    - MIPS: Avoid accidental raw backtrace
    - MIPS: ...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Chris Carlin (crcarlin)]

I'm seeing this in bionic right now. I don't know how to add bionic to the tracker, though, even if xenial is fixed.