Can't get h/w video acceleration on rpi4

[Sergey Borovkov]

Hi, we've run into an issue where we can't get h/w video acceleration working on rpi4 on ubuntu core 18 (and 20).

Our snap is using the following stack:
Qt 5 eglfs + gstreamer + mesa.

Raspberry's config txt has: dt_overlay=vc4-fkms-v3d, gpu_mem=256.

I've enabled all video/opengl related interfaces:
- opengl
- hardware-observe
- framebuffer.

Gstreamer can't detect that it has support for h264 h/w decoding using v4l2 plugin. The very same snap unpacked on the raspbian and that is running in a pretty same way it is on the core immediately detects that h264 decoding is available (i.e. I unsquashfs it there, set the LD_LIBRARY_PATH, WEBGL_DRIVERS_PATH, and all other environment variables that are returned from `env` command when running snap run --shell my_snap).

Because of that Qt falls back to the GStreamer pipeline with s/w decoder (avdec) and outputs like 0.5 frames per second (I am not sure if OpenGL actually works with h/w acceleration because even then it's so slow).

Running on core 20 did not help - I tried running it in devmode as well without any differences.

Raspbian, where it works, has 5.5 version of Linux kernel, I did try the beta version of core 20 which had 5.4 kernel version but it made no difference.

lsmod on raspbian shows some extra modules related to vc4 not present on the core (not sure how relevant they are though):

- v4l2_mem2mem
- bcm2835_codec

[Oliver Grawert (ogra)]

while v4l2_mem2mem is a rather old module from the mainline kernel (and is shiped in our kernels, bcm2835_codec is pi4 specific and should essentially provide access to the decoder, this seems to be missing in our kernel ...

[Oliver Grawert (ogra)]

the config option seems to be:

CONFIG_VIDEO_CODEC_BCM2835=m

[Oliver Grawert (ogra)]

talking to juerg (our kernel maintainer) it seems the codec module should be available in 5.4 in UC20, did you try to manually modprobe it ?

[Sergey Borovkov (serge-borovkov)]

Hi Oliver, sorry, did not mention that the difference in codecs is with core 18 - I only tried the snap on 20 to make sure it does not work there as well. I will check it.

[Sergey Borovkov (serge-borovkov)]

On Core 20:
root@ubuntu:/home/pi# modprobe bcm2835_codec
modprobe: FATAL: Module bcm2835_codec not found in directory /lib/modules/5.4.0-1011-raspi

On raspbian:

sudo modprobe bcm2835_codec
pi@raspberrypi:~ $

Google seems to be telling me that it's an element that provides video encode, decode, and transform functions via V4L2 M2M devices.

Is that a root of this issue?

[Oliver Grawert (ogra)]

this could surely be part of it

[Juerg Haefliger (juergh)]

5.4.0-1011-raspi is an ancient kernel from May. How did you end up using that? Current is 5.4.0-1022.25. With that kernel, I still had to copy the DTBs from the kernel to make it work. So it looks like another DTB/kernel mismatch.

[Sergey Borovkov (serge-borovkov)]

For UC 20 I downloaded the image from here: https://cdimage.ubuntu.com/ubuntu-core/20/beta/current/
I tried refreshing to the edge kernel but then an image would not boot.

For 18 I had built an image from an assertion containing kernel: pi-kernel=18-pi

[Sergey Borovkov (serge-borovkov)]

I guess I will have to build an image with my own assertion, as ubuntu's images are in fact from May and image does not boot if I refresh the kernel. Will update if it does not work once I make it myself.

[Juerg Haefliger (juergh)]

You can try to copy the DTBs (and overlays) from /lib/firmware/5.4.0-<xyz>-raspi/device-tree/broadcom/ to /run/mnt/ubuntu-seed after refreshing the kernel snap.

[Sergey Borovkov (serge-borovkov)]

Thanks, ogra pointed me to the /pending subdirectory for images. I installed the latest one and gstreamer can actually detect v4l2h264 decoding capabilities there.
Unfortunate that it does not work on 18 though.

[Juerg Haefliger (juergh)]

We will fix 18 eventually (when we move the kernel to 5.4). But that requires some snap changes that are currently in the works.

[Viktor Petersson (vpetersson)]

@Juerg - Do you have a timeline for this?

[Juerg Haefliger (juergh)]

This is gated by snapd work. I don't have visibility into the snap team's schedule.

[Maciej Borzecki (maciek-borzecki)]

@juergh do I understand it correctly that this is blocked on snapd pulling dtbs from the kernel snap?

[Maciej Borzecki (maciek-borzecki)]

AFAIU support for pulling in dtbs from the kernel snap is already in snapd. We've recently also fixed a problem with AppArmor profile that blocked access to the GPU device, perhaps that's related. The PR that contains the fix is https://github.com/snapcore/snapd/pull/11822

[Maciej Borzecki (maciek-borzecki)]

Can you please pull the latest pi kernel and gadget from 20/stable, snapd from edge and verify if things are working for you?

[Launchpad Janitor (janitor)]

[Expired for snapd because there has been no activity for 60 days.]