Raspberry Pi 3 microSD support missing from the installer

[William Grant]

[Impact]

Without this module in block-modules.udeb, the default generic arm64 image cannot install on a raspberrypi3 board.

[Test case]

Build the block-modules.udeb with and without the attached patch and check for the presence of the drivers/mmc/host/bcm2835.ko.

[Regression Potential]

None, we are adding a ko to a udeb, not removing it.

---

Please add bcm2835 to block-modules on arm64 (and armhf?).

artful's arm64 d-i mini.iso works fine on Raspberry Pi 3 when booting with a UEFI-capable u-boot, except for one detail: mmc/host/bcm2835.ko is missing from block-modules, so you can't install to microSD without manually grabbing and loading the module.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1729128

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[William Grant (wgrant)]

The linux source package produces the udebs that are installed by d-i.

[Paolo Pisati (p-pisati)]

Your suggestion makes sense, but can you give me the exact steps on how to test it afterward?

I gave a quick glance at u-boot master, but i couldn't find any 'uefi' or 'rpi3-uefi' target in configs. Or shall i just build the rpi3 target and then pass a UEFI fw as a payload? And where do i get the UEFI fw from? Instructions on how to setup the system are welcome.

[William Grant (wgrant)]

On 06/11/17 23:03, Paolo Pisati wrote:
> Your suggestion makes sense, but can you give me the exact steps on how
> to test it afterward?
>
> I gave a quick glance at u-boot master, but i couldn't find any 'uefi'
> or 'rpi3-uefi' target in configs. Or shall i just build the rpi3 target
> and then pass a UEFI fw as a payload? And where do i get the UEFI fw
> from? Instructions on how to setup the system are welcome.

It was lot easier to get working than I had expected. There are a couple
of gotchas left, but it works great (and is the cheapest way I know to
run arm64...) once you get through them... I intend to look at fixing
most of these for bionic.

Prepare an MBR-partitioned microSD card with a FAT16 or FAT32 partition
at the start to be used for firmware and as the UEFI ESP; 100MB is
ample. The only gotcha here is that the firmware doesn't seem to know
about partition type 0xEF (ESP), but u-boot will happily load GRUB from
a 0xE (FAT16 LBA) that the platform firmware is happy with.

Then give it firmware:

  git clone git://git.denx.de/u-boot.git
  pushd u-boot
  make rpi_3_defconfig
  make CROSS_COMPILE=aarch64-linux-gnu-
  # kernel8.img is important -- the 8 forces start.elf to do aarch64.
  cp u-boot.bin /PATH/TO/MICROSD/FAT16_OR_32/PART/kernel8.img
  popd

  git clone --depth 1 gh:raspberrypi/firmware raspberrypi-firmware
  pushd raspberrypi-firmware
  cp bootcode.bin start.elf fixup.dat /PATH/TO/MICROSD/FAT16_OR_32_PART/
  popd

None of the dtbs are required.

Insert the microSD card into the RPi3. Also insert a USB mass storage
device with
http://ports.ubuntu.com/ubuntu-ports/dists/artful/main/installer-arm64/current/images/netboot/mini.iso
written to it.

Boot the RPi3 with Ethernet, HDMI and a USB keyboard. After a few
seconds of u-boot, a hideous looking, bright red, awfully corrupt, but
still readable GRUB 2 prompt will appear. d-i will automatically start,
with no visual issues.

Switch to another VT and grab bcm2835.ko for the relevant kernel. For
artful's 4.13.0-16.19:

  wget --no-check-certificate https://williamgrant.id.au/f/1/2017/bcm2835.ko
  insmod bcm2835.ko

Then switch back to tty0 and proceed with the install. The partitioner
will see the internal microSD, but care is required with partitioning:
if you recreate the partition table it will default to GPT, which the
RPi3 firmware can't read, and if you recreate the boot partition it'll
end up with no firmware and type 0xEF (both of which are fixable
afterwards from another machine). The easiest way is to just set the
existing boot partition as an ESP in the installer, which won't change
the type. Then create a partition for / etc. as, and proceed with the
install as normal.

Everything will now work fine, except nothing will be visible on HDMI
unless GRUB initialises the EFI console, which you can force after the
install by commenting out GRUB_HIDDEN_TIMEOUT in /etc/default/grub and
running update-grub, and you might also need to disable plymouth be
removing "splash".

I haven't tested GPU bits, but the only hardware I found that didn't
work was the WiFi chip, since linux-firmware is missing
brcmfmac43430-sdio.txt (it just has the .bin). But I grabbed a new
brcmfmac4...

[Paolo Pisati (p-pisati)]

I guess i'm doing something wrong here:

1) dd-ed the mini.iso to a usb stick
2) partitioned a microsd card:

$ sudo fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 14,9 GiB, 15931539456 bytes, 31116288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa4f08fe7

Device Boot Start End Sectors Size Id Type
/dev/mmcblk0p1 2048 526335 524288 256M c W95 FAT32 (LBA)

3) built u-boot (v2017.09) using the rpi3_defconfig and copied u-boot.bin as kernel8.img to the vfat partition
4) copied the relevant raspberry-firmware files to the vfat partition

This is how my sd card looks like:

$ ls -la /media/flag/F8EC-BAD1/
total 3228
drwxr-xr-x 2 flag flag 16384 gen 1 1970 .
drwxr-x---+ 3 root root 4096 nov 13 17:31 ..
-rw-r--r-- 1 flag flag 50248 nov 13 15:47 bootcode.bin
-rw-r--r-- 1 flag flag 6551 nov 13 15:47 fixup.dat
-rw-r--r-- 1 flag flag 397344 nov 13 15:43 kernel8.img
-rw-r--r-- 1 flag flag 2820196 nov 13 15:47 start.elf

5) popped the sd card into rpi3 and applied power

6) the board starts booting, executes u-boot but instead of looking for the usb storage, it tries pxe booting, and after several attempts, it fails. (see attached img).

Any chance i'm missing something in uboot boot script? Like some tweaks to uboot env? Or is it supposed to boot from usb storage by default and without any modification?

[William Grant (wgrant)]

Hm, what if you manually "run usb_boot" once PXE fails?

[Paolo Pisati (p-pisati)]

Ok, it appears uboot couldn't recognize my usb pen (it was an old 4GB usb2.0) but worked fine with a more recent 16GB stick, go figure.

After that, I followed all the steps above, and i got to the point of partitioning the disk, but even after inserting the bcm2835 module, i coudln't see any /dev/mmc* device and, indeed, the installer could only find the usb pen as a viable installation media.

Anyhow, i'm not sure how to test it, but here is the bionic generic kernel (and relative debs and udebs) if you want to try it out:

http://people.canonical.com/~ppisati/lp1729128/

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1729128

[Adam Smith (adamsmith)]

Thanks Paolo and Seth for looking into this and the patches. There's a few things that could be done to make life on the pi better. For example, are there any plans to add support for the relatively new pi3b+ to bionic? Adding efi stub to the raspi2 kernels would be great. The upstream and downstream mmc modules are both enabled on the raspi2 kernels and they conflict - I'm going to file a more detailed bug about that later today.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.15.0-19.20

---------------
linux (4.15.0-19.20) bionic; urgency=medium

  * linux: 4.15.0-19.20 -proposed tracker (LP: #1766021)

  * Kernel 4.15.0-15 breaks Dell PowerEdge 12th Gen servers (LP: #1765232)
    - Revert "blk-mq: simplify queue mapping & schedule with each possisble CPU"
    - Revert "genirq/affinity: assign vectors to all possible CPUs"

linux (4.15.0-18.19) bionic; urgency=medium

  * linux: 4.15.0-18.19 -proposed tracker (LP: #1765490)

  * [regression] Ubuntu 18.04:[4.15.0-17-generic #18] KVM Guest Kernel:
    meltdown: rfi/fallback displacement flush not enabled bydefault (kvm)
    (LP: #1765429)
    - powerpc/pseries: Fix clearing of security feature flags

  * signing: only install a signed kernel (LP: #1764794)
    - [Packaging] update to Debian like control scripts
    - [Packaging] switch to triggers for postinst.d postrm.d handling
    - [Packaging] signing -- switch to raw-signing tarballs
    - [Packaging] signing -- switch to linux-image as signed when available
    - [Config] signing -- enable Opal signing for ppc64el
    - [Packaging] printenv -- add signing options

  * [18.04 FEAT] Sign POWER host/NV kernels (LP: #1696154)
    - [Packaging] signing -- add support for signing Opal kernel binaries

  * Please cherrypick s390 unwind fix (LP: #1765083)
    - s390/compat: fix setup_frame32

  * Ubuntu 18.04 installer does not detect any IPR based HDD/RAID array [S822L]
    [ipr] (LP: #1751813)
    - d-i: move ipr to storage-core-modules on ppc64el

  * drivers/gpu/drm/bridge/adv7511/adv7511.ko missing (LP: #1764816)
    - SAUCE: (no-up) rename the adv7511 drm driver to adv7511_drm

  * Miscellaneous Ubuntu changes
    - [Packaging] Add linux-oem to rebuild test blacklist.

linux (4.15.0-17.18) bionic; urgency=medium

  * linux: 4.15.0-17.18 -proposed tracker (LP: #1764498)

  * Eventual OOM with profile reloads (LP: #1750594)
    - SAUCE: apparmor: fix memory leak when duplicate profile load

linux (4.15.0-16.17) bionic; urgency=medium

  * linux: 4.15.0-16.17 -proposed tracker (LP: #1763785)

  * [18.04] [bug] CFL-S(CNP)/CNL GPIO testing failed (LP: #1757346)
    - [Config]: Set CONFIG_PINCTRL_CANNONLAKE=y

  * [Ubuntu 18.04] USB Type-C test failed on GLK (LP: #1758797)
    - SAUCE: usb: typec: ucsi: Increase command completion timeout value

  * Fix trying to "push" an already active pool VP (LP: #1763386)
    - SAUCE: powerpc/xive: Fix trying to "push" an already active pool VP

  * hisi_sas: Revert and replace SAUCE patches w/ upstream (LP: #1762824)
    - Revert "UBUNTU: SAUCE: scsi: hisi_sas: export device table of v3 hw to
      userspace"
    - Revert "UBUNTU: SAUCE: scsi: hisi_sas: config for hip08 ES"
    - scsi: hisi_sas: modify some register config for hip08
    - scsi: hisi_sas: add v3 hw MODULE_DEVICE_TABLE()

  * Realtek card reader - RTS5243 [VEN_10EC&DEV_5260] (LP: #1737673)
    - misc: rtsx: Move Realtek Card Reader Driver to misc
    - updateconfigs for Realtek Card Reader Driver
    - misc: rtsx: Add support for RTS5260
    - misc: rtsx: Fix symbol clashes

  * Mellanox [mlx5] [bionic] UBSAN: Undefined behaviour in
    ./include/linux/net_dim.h (LP: #1...

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Andy Whitcroft (apw)]

This bug was erroneously marked for verification in bionic; verification is not required and verification-needed-bionic is being removed.