USB not working under arm64 on Pi4 with 4G ram

Bug #1848790 reported by Dave Jones on 2019-10-18
180
This bug affects 37 people
Affects Status Importance Assigned to Milestone
linux-raspi2 (Ubuntu)
High
Hui Wang
Eoan
High
Unassigned

Bug Description

Ubuntu 19.10 arm64 on a Raspberry Pi 4 does not recognize a keyboard which works successfully on the same Raspberry Pi 4 with Ubuntu 19.10 armhf. Both USB hubs (2 and 3) were tested, without the OS seeing the keyboard on either. Booting the arm64 image on a Raspberry Pi 3, the keyboard worked successfully.

Output of lsusb under arm64 on a Pi 3:

ubuntu@ubuntu:~$ lsusb
Bus 001 Device 007: ID 413c:2106 Dell Computer Corp. Dell QuietKey Keyboard
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Output of lsusb under armhf on a Pi 4 (same keyboard attached):

ubuntu@ubuntu:~$ lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 413c:2106 Dell Computer Corp. Dell QuietKey Keyboard
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Output of lsusb under arm64 on the same Pi 4 (same keyboard attached):

ubuntu@ubuntu:~$ lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The fact even the VIA Labs hub doesn't show up (which is built in) suggests the kernel is (for some reason) unable to enumerate anything against the USB hubs.

== Temporary Workaround ==

As noted by various people below, the following line can be added to the "usercfg.txt" file on the boot partition:

total_mem=3072

This will limit the available RAM to 3Gb, but otherwise things should operate normally. Note that this bug does not affect Pi 4Bs with less than 4Gb of RAM.

Dave Jones (waveform) wrote :
Download full text (6.8 KiB)

Additional detail from dmesg:

...

[ 1.390148] usbcore: registered new interface driver lan78xx
[ 1.391889] usbcore: registered new interface driver smsc95xx
[ 1.393568] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 1.395273] ehci-pci: EHCI PCI platform driver
[ 1.396956] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 1.398622] ohci-pci: OHCI PCI platform driver
[ 1.400281] uhci_hcd: USB Universal Host Controller Interface driver
[ 1.402251] xhci_hcd 0000:01:00.0: xHCI Host Controller
[ 1.403963] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
[ 1.406172] xhci_hcd 0000:01:00.0: hcc params 0x002841eb hci version 0x100 quirks 0x0000001000000890
[ 1.408067] genirq: irq_chip Brcm_MSI did not update eff. affinity mask of irq 43
[ 1.410313] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.03
[ 1.412064] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.413755] usb usb1: Product: xHCI Host Controller
[ 1.415461] usb usb1: Manufacturer: Linux 5.3.0-1007-raspi2 xhci-hcd
[ 1.417152] usb usb1: SerialNumber: 0000:01:00.0
[ 1.419377] hub 1-0:1.0: USB hub found
[ 1.421083] hub 1-0:1.0: 1 port detected
[ 1.423195] xhci_hcd 0000:01:00.0: xHCI Host Controller
[ 1.424869] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
[ 1.426579] xhci_hcd 0000:01:00.0: Host supports USB 3.0 SuperSpeed
[ 1.428528] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.03
[ 1.430209] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.431906] usb usb2: Product: xHCI Host Controller
[ 1.433577] usb usb2: Manufacturer: Linux 5.3.0-1007-raspi2 xhci-hcd
[ 1.435268] usb usb2: SerialNumber: 0000:01:00.0
[ 1.437425] hub 2-0:1.0: USB hub found
[ 1.439157] hub 2-0:1.0: 4 ports detected
[ 1.441611] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[ 1.443556] dwc_otg fe980000.usb: base=(____ptrval____)
[ 1.647527] Core Release: 2.80a
[ 1.649145] Setting default values for core params
[ 1.650809] Finished setting default values for core params
[ 1.758372] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[ 1.854949] Using Buffer DMA mode
[ 1.856573] Periodic Transfer Interrupt Enhancement - disabled
[ 1.858211] Multiprocessor Interrupt Enhancement - disabled
[ 1.859865] OTG VER PARAM: 0, OTG VER FLAG: 0
[ 1.861530] Dedicated Tx FIFOs mode
[ 1.864183] WARN::dwc_otg_hcd_init:1043: FIQ DMA bounce buffers: virt = ffff0000105d1000 dma = 0x00000000eb500000 len=9024
[ 1.865894] FIQ FSM acceleration enabled for :
               Non-periodic Split Transactions
               Periodic Split Transactions
               High-Speed Isochronous Endpoints
               Interrupt/Control Split Transaction hack enabled
[ 1.874175] dwc_otg: Microframe scheduler enabled
[ 1.874206] WARN::hcd_init_fiq:496: MPHI regs_base at ffff0000102c5200
[ 1.875964] dwc_otg fe980000.usb: DWC OTG Controller
[ 1.877651] dwc_otg fe980000.usb: new USB bus registered, assigned bus number 3
[ 1.879366] dwc_otg fe980000.usb: irq 2...

Read more...

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-raspi2 (Ubuntu):
status: New → Confirmed
Hui Wang (hui.wang) wrote :

I reproduced this bug on my rpi4 board.

Changed in linux-raspi2 (Ubuntu):
importance: Undecided → High
assignee: nobody → Hui Wang (hui.wang)
Ken Howe (leggazoid) wrote :

User on lingon on raspberry pi forum posted the following:

The USB-problem concerning the Raspberry Pi 4GB RAM model might be due to the issue seen earlier that using more RAM than 3072 MB breaks the USB:
https://www.raspberrypi.org/forums/view ... 6&start=25

The issue was solved by a kernel patch:
https://www.raspberrypi.org/forums/view ... 0#p1517839

and the kernel patch was this one:
https://github.com/raspberrypi/linux/is ... -520269280

Ken Howe (leggazoid) wrote :

fixed links

User on lingon on raspberry pi forum posted the following:

The USB-problem concerning the Raspberry Pi 4GB RAM model might be due to the issue seen earlier that using more RAM than 3072 MB breaks the USB:
https://www.raspberrypi.org/forums/v...46766&start=25

The issue was solved by a kernel patch:
https://www.raspberrypi.org/forums/v...rt=50#p1517839

and the kernel patch was this one:
https://github.com/raspberrypi/linux...ment-520269280

Ken Howe (leggazoid) wrote :

fixed links

User on lingon on raspberry pi forum posted the following:

The USB-problem concerning the Raspberry Pi 4GB RAM model might be due to the issue seen earlier that using more RAM than 3072 MB breaks the USB:
[url]https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=246766&start=25[/url]

The issue was solved by a kernel patch:
[url]https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=246766&start=50#p1517839[/url]

and the kernel patch was this one:
[url]https://github.com/raspberrypi/linux/issues/3093#issuecomment-520269280[/url]

Ken Howe (leggazoid) wrote :

Fixed links

User on lingon on raspberry pi forum posted the following:

The USB-problem concerning the Raspberry Pi 4GB RAM model might be due to the issue seen earlier that using more RAM than 3072 MB breaks the USB:
[url]https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=246766&start=25[/url]

The issue was solved by a kernel patch:
[url]https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=246766&start=50#p1517839[/url]

and the kernel patch was this one:
[url]https://github.com/raspberrypi/linux/issues/3093#issuecomment-520269280[/url]

Adam T (tiermann) wrote :

Setting total_mem=2048 in /boot/firmware/usercfg.txt allowed me to boot Pi4 4GB with 19.10 64bit with working USB. Of course I have less RAM now, but working USB.

Likely the above patch will fix it.

Hui Wang (hui.wang) wrote :

Thanks everyone.

I didn't notice this problem just because I set total_mem=1024/2048/3072 before, this is problem was not exposed because the full 4G ram was not enabled.

And I will test the patches of #8 and SRU them if they works well.

thx.

Hui Wang (hui.wang) wrote :

I built a testing kernel, not only includes the fix for USB host, but also includes all new patches from https://github.com/raspberrypi/linux.git rpi-5.3.y branch (about 107 patches).

I tested both arm64 and armhf kernels on Pi4 without HDMI monitor, everything works well.

Could anybody help test these two kernels on Pi4 with HDMI monitor, Pi3 and Pi2 if you have any of them?

After verifying the kernel will not introduce regression on Pi4/3/2, I will submit the patches to UBUNTU kernel.

The new kernel could be downloaded: https://people.canonical.com/~hwang4/rpiv2/

To install and test the kernel:
copy arm64 or armhf folders to rootfs of Pi, sudo dpkg -i linux-modules-xxxx.deb; sudo dpkg -i linux-image-xxx.deb;sudo reboot

thx

Garyschulte (garyschulte) wrote :

Also can confirm that the arm64 image works on raspberry pi 4/4gb, 1080p HDMI. Still no sound device for bcm2835, but USB is happy, I am happy. Thanks for the quick turnaround

Garyschulte (garyschulte) wrote :

Same sdhc with arm64 image and modules work for raspberry pi 3b+/1gb, 1080p HDMI also.

Hui Wang (hui.wang) wrote :

Thanks for your testing.

And I remember we need to enable some configs to make the audio work. If you have time, please have a try with below:

Adding these configs to config.txt

# Enable audio (loads snd_bcm2835)
dtparam=audio=on
disable_audio_dither=on

Rohit Yadav (rohityadav) wrote :

@hui.wang - thanks, seems to work but /dev/kvm is missing. Can you advise why the kernels are not built by default with KVM/VIRTUALIZATION enabled in the kernel config? If this going to change in future.

Hui Wang (hui.wang) wrote :

@Rohit,

I will take a look at the issue of kvm, and do you know if the 18.04 support kvm on rpi3/2?

Rohit Yadav (rohityadav) wrote :

@hui.wang - I'm not sure on RPi3, but Ubuntu 18.04+KVM certainly works on RPi4. See someone already blogged about this here: https://jamesachambers.com/raspberry-pi-4-ubuntu-server-desktop-18-04-3-image-unofficial, I tried their image but it had issues due to older libvirt/qemu versions on 18.04.

Rohit Yadav (rohityadav) wrote :

@hui.wang - I tested the compiled kernel on my RaspberryPi4 (4GB RAM model). Both KVM and external SSD/USB (in my case Samsung T5, 500GB external SSD) on the USB 3.0 port are working fine:

root@cloudstack-mgmt:~# ls /dev/kvm
/dev/kvm
root@cloudstack-mgmt:~# kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

root@cloudstack-mgmt:~# lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 04e8:61f5 Samsung Electronics Co., Ltd Portable SSD T5
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

# fdisk -l
[.. snipped ..]
Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Portable SSD T5
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 976773134 976771087 465.8G c W95 FAT32 (LBA)

root@cloudstack-mgmt:~# blkid -o list
device fs_type label mount point UUID
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
/dev/mmcblk0p1 vfat system-boot /boot/firmware 79A2-EC62
/dev/mmcblk0p2 ext4 writable / 2fae1f20-4a20-4c77-a39b-fb6480f64a0b
/dev/loop0 squashfs /snap/lxd/12210
/dev/loop1 squashfs /snap/core/7920
/dev/sda1 ext4 (not mounted) 91175b3a-ee2c-47a7-a1e5-f4528e127523

This confirms that the patch and kernel-image build by @hui.wang fixes the external USB storage issue with the 4GB RPi4 model. Thanks @hui.wang.

tags: added: id-5db015562d35287792a0a3bc
Dave Jones (waveform) on 2019-10-25
description: updated
wanthalf (wanthalf) wrote :

I also had to add the new revision "Raspberry Pi 4 Model B Rev 1.2" to /usr/share/flash-kernel/db/all.db in order to make flash-kernel work, but that is probably a different "bug".

Hui Wang (hui.wang) wrote :

And also Pi 3 Model A Plus is not in the all.db.

Dave Jones (waveform) wrote :

I've now tested the armhf and arm64 variants of Hui's test kernel from comment #11 on the Pi 2, 3, and 4 (several memory variants) and all seems good so far. Still need to finish testing on the compute modules, though.

As regards flash-kernel, the relevant tickets for the missing entries are LP: #1847587 for the 4B, and LP: #1841955 for the 3A+.

Unfortunately it turns out there's a rather more serious issue with flash-kernel, especially in relation to this bug: it only copies the dtb for the Pi it's being run on to the boot partition. As long as you never switch the SD card between models of Pi that's okay, but to give an example I ran across when testing the fixed kernel here: I upgraded the kernel on a Pi3, tested it, switched the card to a Pi4 and it failed to boot because the Pi4's dtb on the boot partition hadn't been updated (and the fixed kernel has a subtly different dtb). The bug tracking this is #1850678; I've got a fix in the works but it needs a bit of refinement first.

Brad Figg (brad-figg) wrote :

@all

There is a new kernel in -proposed that should fix these USB issues as well as the KVM support. There is still a known issue with audio as is explained in comment #14 above. Please test the -proposed kernel and let us know of any issues that you find.

Changed in linux-raspi2 (Ubuntu Eoan):
status: New → Triaged
importance: Undecided → High
Hadrien Mary (hadim) wrote :

Does someone know when the fix will be ported to the Ubuntu arm64 image?

Hui Wang (hui.wang) wrote :

Maybe after the proposed kernel is verified

To test -proposed kernel, with the old arm64 image:

edit /etc/apt/sources.list
and add below to that file:
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed main restricted
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed universe
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed multiverse

sudo apt-get update

sudo apt install linux-image-5.3.0-1011-raspi2
sudo reboot

Then the image will run the proposed kernel, and users could check it by 'uname -a', it will show 5.3.0-1011

Paulo Matos (pmatos) wrote :

To test this, is there an sd card image of ubuntu for pi4 I can flash?

Hui Wang (hui.wang) wrote :

@Paulo,

Please download (Raspberry Pi 3 (64-bit ARM) preinstalled server image) from http://cdimage.ubuntu.com/releases/eoan/release/

And please note if you use 4G version Pi4 board, the usb keyboard doesn't work with the default kernel in the image, to use the usb keyboard, please add total_mem=3072 in the config.txt, then install the proposed kernel, after that you could remove the total_mem=3072 from the config.txt.

thx.

Tim Crisall (tcrisall) wrote :

Not to clutter up this bug - but some quick feedback:

 - originally installed with mem_total as proposed earlier
 - updated to your first patched kernel as mentioned earlier and removed mem_total
 - updated a day ago to the -proposed version of the kernel

can confirm:

pi4 4G RAM, booting from SD Card, mounting root from USB attached drive, just successfully installed centos8 using KVM...

Thanks!

Adam Mierzwiak (adamvm) wrote :

Confirm that solution from #25 works and total_mem=3072 can be removed from config.

Ian Steiger (isteiger) wrote :

updated my install, removed config.txt entry, available on stable

Changed in linux-raspi2 (Ubuntu Eoan):
status: Triaged → Fix Released
huihui (huihui321) on 2019-11-21
Changed in linux-raspi2 (Ubuntu Eoan):
assignee: nobody → huihui (huihui321)
Hui Wang (hui.wang) wrote :

We are going to release a new kernel soon, right now it is in the proposed channel, Welcome to test with the new kernel:

edit the $rpi_rootfs/etc/apt/sources.list and add:
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed main restricted
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed universe
deb http://ports.ubuntu.com/ubuntu-ports eoan-proposed multiverse

boot the rpi board, then run:
sudo apt-get update
sudo apt install linux-image-5.3.0-1013-raspi2
sudo reboot

Then you could do the test with the proposed kernel.

At least the 1013 kernel fixed these bugs compared with 1012 kernel:
https://bugs.launchpad.net/bugs/1850876
https://bugs.launchpad.net/bugs/1852510

summary: - USB not working under arm64 on Pi4
+ USB not working under arm64 on Pi4 with 4G ram
Unleavened (unleavened) wrote :

Hello, using apt update && apt upgrade the USB already starts working. I see that the last released image for Raspberry Pi 3/4 was on the 12th of November. When can we see a new image released with the fix?

Unleavened (unleavened) wrote :
Hui Wang (hui.wang) 9 hours ago
Changed in linux-raspi2 (Ubuntu Eoan):
assignee: huihui (huihui321) → nobody
Akshay (akshaynakra) wrote :

Hui Wang - I tried your proposed solution. Post rebooting the system freezes at login after I type in my password.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.