v3d driver clock problem forces OpenGL to use software rendering

Bug #1880125 reported by AlpineCarver
66
This bug affects 11 people
Affects Status Importance Assigned to Milestone
linux-raspi (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

release: 32-bit ubuntu 20.04 running on a raspberry pi 4
package: linux-raspi 5.4.0.1008.8

I made the configuration change described in:
 https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1876862
and am running a kernel with v3d on a raspberry pi 4.

Unfortunately, OpenGL is still not using the GPU; it's using the software renderer, llvmpipe.

Setting LIBGL_DEBUG=verbose and running an OpenGL program shows that OpenGL can't open any /dev/dri/renderD* device.

Looking in /dev/dri, there are no device files that begin with "render," indicating the v3d driver isn't starting up properly.

Adding debugging code to the kernel, I find that:
(1) drivers/gpu/drm/v3d/v3d_drv.c:v3d_platform_drm_probe() fails because it can't get a clock - devm_clk_get() returns an error.
(2) in drivers/clk/clk.c:of_clk_get_hw(), the call to of_parse_clkspec() for the v3d driver yields a desired clock source of "firmware-clocks".
(3) drivers/clk/clk.c:of_clk_get_hw_from_clkspec() fails because it can only find six clock providers (aux@7e215000, cprman@7e101000, clock@7ef00000, clk-108M, clk-usb, clk-osc) none of which match the requested "firmware-clocks".

The solution is to add a clock definition to the kernel source and a corresponding change to the device tree, but I'm not familiar enough with either to create that solution myself.

In the raspbian kernel source tree, drivers/clk/bcm/clk-raspberrypi.c defines BCM2835_CLOCK_V3D, but there is no corresponding definition in the ubuntu source tree.

Revision history for this message
AlpineCarver (acarv) wrote :

i forgot to mention: the relevant device tree file is:
    arch/arm/boot/dts/bcm2711-rpi.dtsi

Revision history for this message
AlpineCarver (acarv) wrote :

i'm attaching some small excerpts from dmesg output from some debugging printk calls i added to the kernel v3d and clock code.

there is another small issue with v3d_platform_drm_probe() - when it gets -EPROBE_DEFER from devm_clk_get(), it doesn't return -EPROBE_DEFER to its caller, so v3d_platform_drm_probe() was only being called once. i changed it to return -EPROBE_DEFER, and that resulted in many attempts, but all with the same result - finding 6 clock providers, none of which matches "firmware-clocks".

Revision history for this message
AlpineCarver (acarv) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-raspi (Ubuntu):
status: New → Confirmed
Changed in linux-raspi (Ubuntu Focal):
status: New → Confirmed
Revision history for this message
Hui Wang (hui.wang) wrote :

Oh, the clk_raspberrypi is not enabled in the ubuntu kernel. Let me build a testing kernel to enable it.

Revision history for this message
Hui Wang (hui.wang) wrote :
Revision history for this message
AlpineCarver (acarv) wrote :

comment 33 of bug 1850876 mentions dtoverlay=vc4-fkms-v3d.
i have had that line all along in /boot/firmware/usercfg.txt (which is where it goes on a pi 4).

bug 1850876 references linux-raspi2, and the v2 test kernel is based on linux 5.3.

i'm running ubuntu 20.04 on a pi 4 - the raspi branch - which is based on linux 5.4.

do you have a test kernel built from the raspi branch, or is there a git commit i can check out to build it myself?

i have been successfully building and running kernels from source obtained via:
git clone --depth=1 git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal -b raspi

Revision history for this message
Rocky Stone (rstone-mail) wrote :

Looks like this commit has already enabled CONFIG_CLK_RASPBERRYPI: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/commit/?h=raspi&id=5be0baf1f125006ba31157316d8aef25cbb00f5b with much the same reason as described in this bug

Revision history for this message
AlpineCarver (acarv) wrote :

That commit was 9 weeks and 422 commits ago. At some point after that, it was disabled again.

There have been a number of commits that mention v3d since then, including several that mention clocks.

Is it sufficient simply to re-enable CONFIG_CLK_RASPBERRYPI?

Revision history for this message
Rocky Stone (rstone-mail) wrote :

> That commit was 9 weeks and 422 commits ago. At some point after that, it was disabled again.
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/tree/arch/arm64/configs/bcm2711_defconfig?h=raspi
Well in the latest version https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/tree/arch/arm64/configs/bcm2711_defconfig?h=raspi I see that both 'CONFIG_CLK_RASPBERRYPI' and 'CONFIG_ARM_RASPBERRYPI_CPUFREQ' are set to 'y' why do you think it was disabled again?

> Is it sufficient simply to re-enable CONFIG_CLK_RASPBERRYPI?
I don't know, but the original commit also enabled CONFIG_ARM_RASPBERRYPI_CPUFREQ=y

If Hui Wang meant with his comment#7[1] something different than enabling 'CONFIG_CLK_RASPBERRYPI' and 'CONFIG_ARM_RASPBERRYPI_CPUFREQ' I don't know what it is.

[1] https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1880125/comments/7

Revision history for this message
AlpineCarver (acarv) wrote :

> why do you think it was disabled again?
I was looking in .../focal/debian.raspi/config/...

Investigating this bug is my first foray into the linux kernel source; I'm not familiar with the details of kernel configuration.

So if it's already enabled in .../focal/arch/arm64/configs/... and has been all along, then I guess it's not the source of the current problem.

Revision history for this message
AlpineCarver (acarv) wrote :

Re-examining the output of the debug code I added a few weeks ago, I noticed that I had put a kprintf into clk-raspberrypi.c/raspberrypi_clk_probe(), and it never was invoked.

Thus, I think you're right that adding:

CONFIG_CLK_RASPBERRYPI=y
CONFIG_ARM_RASPBERRYPI_CPUFREQ=y

has a good chance of getting us past this problem.

I will try adding those lines to .../focal/debian.raspi/config/config.common.ubuntu. I'm having trouble building/running a modified kernel at the moment, but will do this as soon as I can.

Revision history for this message
AlpineCarver (acarv) wrote :

I have now built and run a kernel with:

CONFIG_DRM_V3D=m
CONFIG_CLK_RASPBERRYPI=y
CONFIG_ARM_RASPBERRYPI_CPUFREQ=y

in .../focal/debian.raspi/config/config.common.ubuntu and i can confirm that it fixes the problem.

My debug code shows raspberrypi_clk_probe() successfully adding clock "firmware-clocks" and v3d_platform_drm_probe() now succeeding.

/dev/dri/renderD128 exists.

A simple opengl test program renders correctly and reports:
    renderer = V3D 4.2
whereas before this change, it reported:
    renderer = llvmpipe

Note that this is a 32-bit kernel, based on the code in the raspi branch, which is at kernel version 1009. I don't know if the same changes will also work in 64-bit kernels. A recent post in raspberry pi forums suggests it may not.

Revision history for this message
AlpineCarver (acarv) wrote :

I'll try 64-bit later today.

Revision history for this message
Attila T. Áfra (attila-afra) wrote :

I had the same issue with the latest stable 64-bit kernel. However upgrading to the latest proposed 5.4.0-1014.14 kernel (https://launchpad.net/ubuntu/+source/linux-raspi/5.4.0-1014.14) fixed it, and now the V3D driver works by default. I tried only with 64-bit, on a Raspberry Pi 4 Model B Rev 1.4.

Revision history for this message
AlpineCarver (acarv) wrote :

Great!

A 64-bit kernel I built with the same changes also works.

Revision history for this message
Petr Vodička (vodickape) wrote :

I have the same issue . My kernel obtained by yesterday´s upgrade is:
Linux ubuntu 5.4.0-1013-raspi #13-Ubuntu SMP Mon Jun 15 03:17:37 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
SW acceleration still does not run:
ubuntu@ubuntu:~$ glxinfo -B
name of display: :0
Accelerated: no

When could we expect orinary automatic upgrade to 5.4.0-1014.14 kernel which fix this issue ?
Thanks

Revision history for this message
Juerg Haefliger (juergh) wrote :

You can enable the proposed archive to pull it in via apt upgrade or wait a couple of days until our regression testing finishes and it gets promoted to updates.

Revision history for this message
AlpineCarver (acarv) wrote :

Just checking in to see if there's any ETA on promoting this fix into focal-updates for the pi 4.

Revision history for this message
AlpineCarver (acarv) wrote :

i can confirm that this bug is FIXED in the 1015 kernel which i downloaded through the normal software update process.

tested on both 32-bit and 64-bit kernels.

THANKS!

Revision history for this message
AlpineCarver (acarv) wrote :

I have verified that the bug is fixed in the most recent software update on both 32-bit and 64-bit installations of ubuntu 20.04 on a pi 4.

Changed in linux-raspi (Ubuntu Focal):
status: Confirmed → Fix Released
Revision history for this message
AlpineCarver (acarv) wrote :

I have marked the "Focal" context for this bug as "Fix Released."

Is observing it fixed on Focal sufficient to close it for the "linux-raspi (Ubuntu)" context?

Revision history for this message
lesar (leonardo-saracini) wrote :

I'm on rpi 4b 8GB, Kernel: 5.4.0-1018-raspi

but still not working:

glxinfo -B
name of display: :1
display: :1 screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: VMware, Inc. (0xffffffff)
    Device: llvmpipe (LLVM 10.0.0, 128 bits) (0xffffffff)
    Version: 20.0.8
    Accelerated: no
    Video memory: 7811MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 3.3
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: llvmpipe (LLVM 10.0.0, 128 bits)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 20.0.8
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.1 Mesa 20.0.8
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 20.0.8
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

tags: added: focal
Changed in linux-raspi (Ubuntu):
status: Confirmed → Fix Released
no longer affects: linux-raspi (Ubuntu Focal)
Revision history for this message
Yonggang Luo (luoyonggang) wrote :

root@ubuntu:/opt/work/cuttlefish# uname -a
Linux ubuntu 5.4.0-1022-raspi #25-Ubuntu SMP PREEMPT Thu Oct 15 13:31:49 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
root@ubuntu:/opt/work/cuttlefish# ls /dev/dri
ls: cannot access '/dev/dri': No such file or directory
root@ubuntu:/opt/work/cuttlefish#

This version also doesn't have dri

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Please try the workaround from bug 1896164.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.