Web browsers lacking hardware-accelerated video decoding

[Michel-Ekimia]

This is a Meta Bug for GPU accelerated video decoding on Ubuntu/Linux browsers.
We will close this bug when most browsers will have Va-api decoding working OOTB without user interaction like in WinOS and MacOS

January 2024:

- Quickest answer : Install firefox Flatpak https://flathub.org/fr/apps/org.mozilla.firefox
 activate VAAPI with the flag media.ffmpeg.vaapi.enabled=true

- Snap : Test in Progress
- deb from mozilla : not tested.

August 2022 :

* Ubuntu 22.04 : Launch firefox snap on Intel Tiger lake -> Check with intel_gpu_top that no codec are GPU decoded ( either VP9 , H264 or AV1 )

   -> change media.ffmpeg.vaapi.enabled to true = VP9 and H264 decoding works
   -> AV1 hwdec does not work in Firefox or VLC , but does with MPV . We need to fix this as most youtube is AV1 now.

* In july 2020, things are getting better :

- Chromium :

A patch is used on most distro packages. , more info here https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1816497

- Firefox :

Firefox team has done a great job and now Va-api decoding in Wayland is possible and X11 is possible in nightly !

Follow it here : https://bugzilla.mozilla.org/show_bug.cgi?id=1210727

* In 2015 I opened this bug , no easy solution was available and battery drained when playing video in browser was crazy.

[ryan14 (ryan14)]

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100401 SUSE/3.6.3-1.2 Firefox/3.6.3
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100401 SUSE/3.6.3-1.2 Firefox/3.6.3

There should be a feature that enables HTML5 video to use GPU acceleration so this will take stress off the CPU. It should work on Ubuntu, OpenSuse, Fedora, etc, and it should work with all graphics cards including onboard video.

Reproducible: Always

[Mardeg-5 (mardeg-5)]

This may depend on or be a dupe of bug 556027

[Jo-hermans (jo-hermans)]

This is filed before, and even partially implemented (see bug 555839 for instance), but there's still lots of work to do. Especially on the cross-platform behaviour (most work is done in the windows side at the moment).

[Nickolay Ponomarev (asqueella)]

Also see bug 495727.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michel-Ekimia (michel.ekimia)]

Hi chad , For few versions chromium was unable to use the vaapi , It's now possible again using the patch from the dev branch :

https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev

Can you integrate this patch ?

Also a lot of GPU VP8/VP9 decoding become possible with nvidia and Intel cards so that will be interresting to test.

[N. W. (nw9165-3201)]

Hello,

I was surprised to see that GStreamer is no longer needed for video playback and that only FFmpeg/Libav is needed now:

https://bugzilla.mozilla.org/show_bug.cgi?id=1207429

Which is very nice.

But I have a question:

There's a bug regarding hardware accelerated video playback with GStreamer, see:

https://bugzilla.mozilla.org/show_bug.cgi?id=894372

So, I was hoping that Firefox would support fully hardware accelerated video playback now that GStreamer is no longer required and went ahead and installed the latest Firefox Nightly 44.0a1 on Ubuntu 15.04 to test it.

Video playback indeed was working without GStreamer, but the video playback still didn't seem to be fully hardware accelerated ;(.

Why is that?

Why is the video playback on Linux still not fully hardware accelerated :(?

Regards

[[e-mail] (sheepdestroyer-gmail)]

I'm on Fedora 23, ffmpeg and vaapi libs installed. Intel Sandy-Bridge hd3000. HW decoding with libva works in native apps.

Testing FF 45 nightly , in about:support I see : "Supports Hardware H264 Decoding No;"

Is there a way to test that ffmpeg is in fact used ? And for for HW accel?

[Lhenry (lhenry)]

Jean-Yves, can you help answer?
We could also maybe document this on SUMO.

[Jyavenard-9 (jyavenard-9)]

There is no unique nor "official" software path to have hardware acceleration on linux.
Every chip makers have designed their own framework to do so; none of them compatible with one another.

You may want to track bug 1210729 or bug 1210727.

The only GStreamer plugin allowing hardware acceleration decoding was a VAAPI plugin: it was buggy and extremely unstable nor did it provide any significant speed improvement.

You're much better off using ffmpeg

[Oartin (oartin)]

As far as I know, there is 2 main distinct framework for hardware acceleration on linux - VDPAU and VAAPI. Eg. VLC or Mplayer supports this, but not Firefox.

I read some comments, that GStreamer + VAAPI plugin works for someone in Firefox, but personally I could never manage it to work (video plays, but not HW accelerated), move to ffmpeg seems to be a good decision.

As far as I know, VLC also uses ffmpeg internally and is able to use HW acceleration frameworks, so I suppose there should be a way for Firefox to use it also.
Now video playback in Firefox on Linux means quite high CPU usage (for me - fullHD H.264 HTML5 video on recent Broadwell i7 CPU takes ~130% of CPU - more than entire one CPU core).

[Tony Houghton (h-realh)]

(In reply to Jean-Yves Avenard [:jya] from comment #7)
> There is no unique nor "official" software path to have hardware
> acceleration on linux.
> Every chip makers have designed their own framework to do so; none of them
> compatible with one another.

That isn't really true, or at best several years out of date. There are effectively only two frameworks, VDPAU (supported by NVidia's proprietary drivers and adopted by the open source NVidia and AMD drivers) and VAAPI (for Intel). Each can use the other as a back-end if necessary, but as far as I know you could be right that it doesn't work well. AMD's proprietary driver supposedly has bindings for something else, but they never provided any support or documentation, so that can be ignored.

> You may want to track bug 1210729 or bug 1210727.
>
> The only GStreamer plugin allowing hardware acceleration decoding was a
> VAAPI plugin: it was buggy and extremely unstable nor did it provide any
> significant speed improvement.

VAAPI is a little behind VDPAU in terms of maturity and take-up, but both now work well in every other significant Linux video player. I heard the reason it was slow in Firefox was because they made it render into main RAM and copied the data to the framebuffer entirely in software. Use VAAPI properly and the CPU usage barely registers.

> You're much better off using ffmpeg

Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy for Firefox to use that route to gain GPU acceleration.

[Z-tim-u (z-tim-u)]

For what it's worth, embedded systems / SoC vendors often provide support for hardware-accelerated video decoding via OpenMax (omx), in addition to any vendor-specific APIs that may also exist.

[Jyavenard-9 (jyavenard-9)]

(In reply to Tony Houghton from comment #9)
> > You're much better off using ffmpeg
>
> Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy
> for Firefox to use that route to gain GPU acceleration.

This is irrelevant to what ffmpeg does, nor if it was compiled to support either vaapi or vdpau. Compiling FFmpeg with vdpau or vaapi support won't automagically make applications using ffmpeg start using hardware acceleration unfortunately.

The P in VDPAU stands for Presentation. The only way you can paint an image decoded with VDPAU is to *P*resent it directly using VDPAU. It can't be rendered the way all the other images are typically painted.
VAAPI is similar (There's also the issue that an entire generation of intel GPUs are buggy)

We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have no other options but copying the decoded image out of the GPU memory into a software surface (which is exactly what the vaapi gstreamer plugin does).

Doing so almost always annihilate any benefits, as copying the image back from the GPU is typically slow (and even more so with nvidia devices)

To make things worse, at this stage, firefox on linux doesn't have hardware accelerated compositor either (OpenGL isn't enabled by default).

We won't be using FFmpeg's hwaccel layer to access vaapi or vdpau (it's only a helper)

(BTW, AMD's own XvBA is perfectly documented)

[Jyavenard-9 (jyavenard-9)]

(In reply to t.i.m from comment #10)
> For what it's worth, embedded systems / SoC vendors often provide support
> for hardware-accelerated video decoding via OpenMax (omx), in addition to
> any vendor-specific APIs that may also exist.

Yes. We are currently developer an OpenMax decoder (bug 1224887). This is for our own FirefoxOS devices, and I'm not sure how easy it will be to enable on other platforms, but the majority of the work will be there.

[Oartin (oartin)]

(In reply to Jean-Yves Avenard [:jya] from comment #11)
> (In reply to Tony Houghton from comment #9)
> VAAPI is similar (There's also the issue that an entire generation of intel GPUs are buggy)

Maybe I haven't noticed this issue, but I can say, that I didn't noticed problems on Ivy Bridge, Haswell and Broadwell GPUs (VLC with VAAPI works just fine).

> We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have
> no other options but copying the decoded image out of the GPU memory into a
> software surface (which is exactly what the vaapi gstreamer plugin does).
>
> Doing so almost always annihilate any benefits, as copying the image back
> from the GPU is typically slow (and even more so with nvidia devices)

How is the situation different on Windows? Just trying to get, why is it such a problem.

> To make things worse, at this stage, firefox on linux doesn't have hardware
> accelerated compositor either (OpenGL isn't enabled by default).

OpenGL on linux can be enabled manually, I'm using it for months without apparent problems.
GPU Accelerated Windows: 1/1 OpenGL (OMTC) (in about:support)

If OpenGL compositor on Linux will be working without problems, would be still needed to copy image from GPU to software, or can it be avoided? If I remember right, I read some article about gstreamer+vaapi in Firefox and it stated, that this copying is killing VAAPI performance.

I'm just trying to figure out, what is needed to get VAAPI/VDPAU working...

[Jyavenard-9 (jyavenard-9)]

(In reply to martin from comment #13)

> > Doing so almost always annihilate any benefits, as copying the image back
> > from the GPU is typically slow (and even more so with nvidia devices)
>
> How is the situation different on Windows? Just trying to get, why is it
> such a problem.

On windows we go through the Windows Media Foundation framework for which the intel driver provides a hardware decoder and plug in automatically to WMF.

So the decoder returns a DXVA surface, which we can render immediately on the screen as we have a DXVA and D3D compositor.

On linux, there's no such common framework.

> OpenGL on linux can be enabled manually, I'm using it for months without
> apparent problems.
> GPU Accelerated Windows: 1/1 OpenGL (OMTC) (in about:support)
>
> If OpenGL compositor on Linux will be working without problems, would be
> still needed to copy image from GPU to software, or can it be avoided?

VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then be drawn without having to first copy it to software.
Though, IIRC, Intel was talking about deprecating that API so that you must go through the VAAPI framework to draw the surface. If they did that, we would have to do the same work as we need for VDPAU

> If I remember right, I read some article about gstreamer+vaapi in Firefox and it
> stated, that this copying is killing VAAPI performance.
this is what I stated earlier on !

>
> I'm just trying to figure out, what is needed to get VAAPI/VDPAU working...

So for VDPAU, we need a VDPAU compositor. Something that would allow us to draw a VDPAU surface into the screen.
The way VDPAU works typically is that you create an OpenGL surface, assign it a colorkey and then tell VDPAU to draw on that color key. That allows to draw the VDPAU surface into the OpenGL surface and then render that surface on the screen using OpenGL.
It's similar to your blue screen stuff used on TV and movies.

Those are all things that need to be done if we want fully accelerated hardware decoding. Where we use the GPU to decode a video frame, and stay within the GPU to render it on the screen.
Right now, we can't.

[Tony Houghton (h-realh)]

(In reply to Jean-Yves Avenard [:jya] from comment #11)
> (In reply to Tony Houghton from comment #9)
> > > You're much better off using ffmpeg
> >
> > Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy
> > for Firefox to use that route to gain GPU acceleration.
>
> This is irrelevant to what ffmpeg does, nor if it was compiled to support
> either vaapi or vdpau. Compiling FFmpeg with vdpau or vaapi support won't
> automagically make applications using ffmpeg start using hardware
> acceleration unfortunately.

I know that, but doesn't ffmpeg provide some sort of API wrapper so that if you're already using it for software decoding it's easier to adapt your code for acceleration instead of starting again from scratch? And even if not, its demuxers and audio decoders should still be useful alongside VDPAU/VAAPI.

> The P in VDPAU stands for Presentation. The only way you can paint an image
> decoded with VDPAU is to *P*resent it directly using VDPAU. It can't be
> rendered the way all the other images are typically painted.
> VAAPI is similar (There's also the issue that an entire generation of intel
> GPUs are buggy)

That's just one type of GPU, there are many others that could be supported without that sort of hardware/driver problem.

> We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have
> no other options but copying the decoded image out of the GPU memory into a
> software surface (which is exactly what the vaapi gstreamer plugin does).
>
> Doing so almost always annihilate any benefits, as copying the image back
> from the GPU is typically slow (and even more so with nvidia devices)
>
> To make things worse, at this stage, firefox on linux doesn't have hardware
> accelerated compositor either (OpenGL isn't enabled by default).

But OpenGL can be enabled, and Firefox's support for WebGL on Linux seems quite good. Seeing as you can use an accelerated surface for WebGL canvases, is it really that much harder to set one up for video elements so you can use VDPAU or VAAPI effectively?

I really can't agree that we're better off with software-only ffmpeg and that you shouldn't make every effort with GPU decoding. I've experienced that even some very recent generation desktop CPUs (eg Core i3 @ 3GHz) can't maintain the proper frame rate with some HD video. And 4K videos are already appearing on youtube. (Although we'd presumably have to wait for Firefox's DASH support for these).

> We won't be using FFmpeg's hwaccel layer to access vaapi or vdpau (it's only
> a helper)
>
> (BTW, AMD's own XvBA is perfectly documented)

Nevertheless, there seems to be something blocking its adoption. But I think there was talk of exposing it via VAAPI if and when it does get supported properly, anyway.

[antistress (antistress)]

Moreover hardware accelerated decoding seems to fit with project candle (rather than software decoding)

[Jyavenard-9 (jyavenard-9)]

(In reply to Tony Houghton from comment #15)

> I really can't agree that we're better off with software-only ffmpeg and
> that you shouldn't make every effort with GPU decoding. I've experienced

What I said was that you're better of using ffmpeg over gstreamer with the vaapi plugin
Memory bandwidth with 4K quickly becomes the bottleneck. On windows in a current work in progress simply removing a single frame copy sped up the decoding frame rate by 6%; this is huge.

At not time did I state that we didn't want to do full GPU acceleration.

Just that things aren't trivial as what you appear to think. Why do you think chrome dropped it entirely very recently?
If it was that easy and worked well, it would be done already.

[Tony Houghton (h-realh)]

(In reply to Jean-Yves Avenard [:jya] from comment #17)
> (In reply to Tony Houghton from comment #15)
>
> > I really can't agree that we're better off with software-only ffmpeg and
> > that you shouldn't make every effort with GPU decoding. I've experienced
>
> What I said was that you're better of using ffmpeg over gstreamer with the
> vaapi plugin
> Memory bandwidth with 4K quickly becomes the bottleneck. On windows in a
> current work in progress simply removing a single frame copy sped up the
> decoding frame rate by 6%; this is huge.
>
> At not time did I state that we didn't want to do full GPU acceleration.

OK. I just got the impression you weren't interested in supporting it. I hope you do manage to get it working.

> Just that things aren't trivial as what you appear to think. Why do you
> think chrome dropped it entirely very recently?

Have they dropped it altogether, even for ChromeOS, now? I know they disabled it for Linux, but didn't remove it before, and Chromium could be recompiled with it enabled with a few changes to some #if statements etc. True, it has been very unreliable, but in the sense that it never works in some versions/systems and falls back to software rather than in the sense that it keeps crashing; when it does work it's stable enough. But the reason they gave for disabling it in Linux was that they had nobody to work on it, not that they couldn't get it working.

> If it was that easy and worked well, it would be done already.

It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to support it together with all the other things a browser has to do makes it harder, but surely not bordering on impossible, and I'm sure at Mozilla you have the programming talent to make it work.

[Jyavenard-9 (jyavenard-9)]

(In reply to Tony Houghton from comment #18)

> It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to

well, they have a single window to worry about. I got it working just fine in mythtv too (as well as vdpau).
They don't have to worry about compositing with any other objects, images or handle someone who has decided to make the video element move around in an html page.

[sreerenj b (bsreerenj)]

(In reply to Jean-Yves Avenard [:jya] from comment #14)
> (In reply to martin from comment #13)

>
> VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then
> be drawn without having to first copy it to software.
> Though, IIRC, Intel was talking about deprecating that API so that you must
> go through the VAAPI framework to draw the surface. If they did that, we
> would have to do the same work as we need for VDPAU

If you are talking about this api:http://cgit.freedesktop.org/vaapi/libva/tree/va/egl/va_egl.h,
Then yes, it is not implemented in driver.

But there are other ways to map deocoded va sufaces to eglimage through dmabuf and vaAcquireBufferHandle.

A detailed implementation with comments here: https://github.com/01org/gstreamer-vaapi/blob/master/gst-libs/gst/vaapi/gstvaapisurface_egl.c

Anther implementation for the same here: https://github.com/01org/libyami/blob/master/egl/egl_vaapi_image.cpp

[Oartin (oartin)]

(In reply to Jean-Yves Avenard [:jya] from comment #14)
> (In reply to martin from comment #13)
> VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then
> be drawn without having to first copy it to software.
> Though, IIRC, Intel was talking about deprecating that API so that you must
> go through the VAAPI framework to draw the surface. If they did that, we
> would have to do the same work as we need for VDPAU

> So for VDPAU, we need a VDPAU compositor. Something that would allow us to
> draw a VDPAU surface into the screen.
> The way VDPAU works typically is that you create an OpenGL surface, assign
> it a colorkey and then tell VDPAU to draw on that color key. That allows to
> draw the VDPAU surface into the OpenGL surface and then render that surface
> on the screen using OpenGL.
> It's similar to your blue screen stuff used on TV and movies.

Ok, thank you for explaining.
So, if I understood correctly, the blocker now is the OpenGL compositor, which is now disabled by default on linux. I guess there are some reasons for that, some issues?
When OpenGL compositor will be working fine, then it could be possible to use VAAPI (using that - maybe in future deprecated - API to map VAAPI surface to OpenGL surface) - that would be the (relatively) easy way, right?

In fact I'm not sure, what VDPAU/VAAPI compositor means - does it mean entire new compositor (like the OpenGL one, which probably still has some issues (guessing from it's not enabled by default)), or is it some "overlay" on OpenGL compositor?
The "entirely new" option would be bad, I'm afraid it would take very long time... It's really a problem on linux, I also experienced lags and dropped frames in some HD videos, so I hope it wouldn't take years to support video HW acceleration.

[Tony Houghton (h-realh)]

(In reply to Jean-Yves Avenard [:jya] from comment #19)
> (In reply to Tony Houghton from comment #18)
>
> > It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to
>
> well, they have a single window to worry about. I got it working just fine
> in mythtv too (as well as vdpau).
> They don't have to worry about compositing with any other objects, images or

Actually they do, most of those other players can overlay GUIs etc on the video.

> handle someone who has decided to make the video element move around in an
> html page.

Ick. But can't they also do that with canvases, so you would already have come up with some solution for WebGL?

[Ajones-m (ajones-m)]

(In reply to Tony Houghton from comment #22)
> (In reply to Jean-Yves Avenard [:jya] from comment #19)
> > handle someone who has decided to make the video element move around in an
> > html page.
>
> Ick. But can't they also do that with canvases, so you would already have
> come up with some solution for WebGL?

Think about scrolling a video on a touch screen. The video needs to scroll coherently with the rest of the image as well as be able to support pinch-zoom. This becomes more difficult if the video is presented through a parallel presentation path as it would be with VDPAU. Ultimately the accelerated decoding on Linux doesn't match with Firefox's compositor model. In order to support accelerated decoding on Linux, someone needs to write some code to bridge that gap. It is not simply a matter of enabling acceleration.

> > They don't have to worry about compositing with any other objects, images or
>
> Actually they do, most of those other players can overlay GUIs etc on the
> video.

You are ignoring the simple fact that video players are architecturally simpler than browsers. Video players have no other purpose than playing video.

We are definitely interested in getting hardware acceleration to work on Linux. If it is as easy as you seem to think it is and you've got some spare time on your hands then it might be a fun project for you.

[Tony Houghton (h-realh)]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #23)
> (In reply to Tony Houghton from comment #22)
> >
> > Ick. But can't they also do that with canvases, so you would already have
> > come up with some solution for WebGL?
>
> Think about scrolling a video on a touch screen. The video needs to scroll
> coherently with the rest of the image as well as be able to support
> pinch-zoom. This becomes more difficult if the video is presented through a
> parallel presentation path as it would be with VDPAU. Ultimately the
> accelerated decoding on Linux doesn't match with Firefox's compositor model.
> In order to support accelerated decoding on Linux, someone needs to write
> some code to bridge that gap. It is not simply a matter of enabling
> acceleration.

I know, but what I was saying is that you already have a way of dealing with specialised rendering surfaces for WebGL canvases. What's so different about VDPAU and VAAPI surfaces? Or does Firefox's WebGL implementation have the same issue that the gstreamer-vaapi implementation did with inefficient copying from an off-screen GPU buffer to the window surface?

> > > They don't have to worry about compositing with any other objects, images or
> >
> > Actually they do, most of those other players can overlay GUIs etc on the
> > video.
>
> You are ignoring the simple fact that video players are architecturally
> simpler than browsers. Video players have no other purpose than playing
> video.

They may not have to deal with scrolling and clipping, but some of them can do a lot more. Kodi has a very sophisticated OSD, supporting non-rectangular transparency and arbitrary downloaded images. It can run full-screen or in a window and either play the video taking up the entire window/screen, or just part of it.

But I take your point, it would have been designed as a video player that has to fit static elements over and around the video, which is back-to-front compare to a browser.

> We are definitely interested in getting hardware acceleration to work on
> Linux.

Thank you for making an effort. I got the wrong impression to start with, because I'm more used to the GNOME bugzilla, where developers pointing out difficulties is usually their making excuses to ignore important issues for years. Ironically though, their browser does seem to support VAAPI at least, but of course it's a long way behind Firefox in other areas.

> If it is as easy as you seem to think it is and you've got some spare
> time on your hands then it might be a fun project for you.

It wouldn't be an efficient use of developer time. I don't mean that I've got better things to do, but that you probably have better people for the job. I don't already know how to use VDPAU and/or VAAPI anyway, but even if I did, I think it would be easier for Firefox developers to learn VDPAU/VAAPI than a VDPAU/VAAPI expert to learn their way around Firefox's code base.

[agm97 (albertogomezmarin)]

i am new in the bugzilla, hello to all the world. i want to say that with archlinux all plugins installed, firefox nightly for testing too, nothing have accelerated video rendering(and my intel integradted is capable for that) I know that you, all the develops have much work.. but.. it seems that windows have better hardware capabilities, and not only in this things is better, in the html5 support too I think, boys.. please.. in Linux the predetermined browser is firefox while in windows google chrome is the most used, chrome/chromium develops aren't interested in hardware acceleration either for linux..I think is the time for the change..not only for this request, but for all request of linux that windows have implemented now

[agm97 (albertogomezmarin)]

and don't tell me the drivers isn't good enoug because intel drivers are very well, amd open source too, and nvidia propietary drivers too, create a black list with amd cloused drivers and nvidia open drivers andfixed for the stablity

[Turbulizator721 (turbulizator721)]

I encourage everyone here. It is the only remaining flaw in Linux

[Oartin (oartin)]

Aparently there is low to none interest from devs about this issue, it's been almost 3 months without any response.

Meanwhile I tried Chromium, that can support VAAPI (well not oficially, but VAAPI support is there, it has to be patched to work not only in Chrome OS). Since Chromium devs aren't interested in supporting linux, it's not really straightforward to get it to work, but I've made it and HW acceleration is working (CPU usage drops in video playback). I tried it only to be sure it's working elsewhere (and that it could be done in Firefox too).

VAAPI also works well in normal video players like VLC (on Intel Broadwell IGPU), so I don't think buggy drivers is valid reason for not implementing that. There could be configuration option to turn HW accel off, if on some machine it's causing a problem.

It's a shame that Firefox devs don't care much about Linux too. Firefox is imo still most used browser in Linux and video playback with Firefox is really a pain.
It's really frustrating that even with new Intel Broadwell i7, playback of 1080p video causes high CPU usage (thus low life time on battery) and that the video is still not fluent, it still shutters/lags.

I don't know what else to say. I think it's one of most serious issues of Firefox in Linux now.
Are there any developers interested in this, are there any plans about fixing this issue sometime in near future?

[RussianNeuroMancer (russianneuromancer)]

Seems like this issue also affects Opera users.

[RussianNeuroMancer (russianneuromancer)]

Saikrishna Arcot clarified that this feature request doesn't affect Opera users.

[dino99 (9d9)]

hi Michel
i understand the general frustation, but chromium is not the only choice after all. Its clear that nothing will be done to support that feature on linux (pure management resources decision)

[Olivier Tilloy (osomon)]

Tentatively assigned myself to re-assess the situation, and try to understand if enabling VAAPI support is doable with a distro-patch, and how much work that would represent.

[W-jan-k (w-jan-k)]

"The Quantum Flow (QF) initiative is a cross-component effort to improve user perceived responsiveness and performance."
Could you please triage this if it would meet your demands for 57 or 58?

Video hardware decoding is very desirable for Linux devices that:
* do not have a powerful CPU
* should be energy-efficient (laptops)
* want to fluently play 4K videos on YouTube while using the CPU for compiling Nightly. ;-)

There is new code from Chromium in comment 17 providing VAAPI hardware video decoding that could and should be shared, or not?
If Mozilla and Google both would fix bugs it would result in far better stability for all users. Would it be possible?
At least we have example code now.

[W-jan-k (w-jan-k)]

(In reply to Jean-Yves Avenard [:jya] from comment #15)
> Yes, we also need accelerated layers to be able to render the frames.

Could we say we have a dependency to WebRender (instead of the current painting code) here, to prevent double work?

[Erdem U. Altinyurt (spamjunkeater)]

Personally, I do NOT care about accelerated layers for rendering frames. Who cares?
I just want to use "accelerated video decoding via HW" because solving the video codec exhaust many CPUs, not rendering frames after. Yes rendering via OGL will probably give much better results but who cares? We need accelerated youtube videos ASAP.

Also, read the first reply of :jya at the topic,
"for just decoding, not rendering (yet)."

I believe, after 2 years, devs need to complete first step using current painting stack. Because this gonna hurt firefox project much more than other bugs. Since linux users cannot watch simple videos with FF, they gonna switch to Chrome soon.

[Michel-Ekimia (michel.ekimia)]

@dino99 : are you working at google on chromium ?

Your argument is pretty wrong, there is Zero impact if this feature is hidden with a flag that one could enable when deployed.

Today we are stuck because we need to recompile.

Sorry but one could think that google wants to slow DesktopLinux progress.

[sam tygier (samtygier)]

There is a bounty for this issue at https://www.bountysource.com/issues/55506502-add-va-api-hardware-decoding-support-on-linux

[Jonathan Watt (jwatt)]

Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)

[Michel-Ekimia (michel.ekimia)]

Hi olivier , can we see if the Ubuntu package can include the patch at least for 18.04 ? We talk about version 66 which is available now.

[Olivier Tilloy (osomon)]

The current upstream effort is tracked at https://chromium-review.googlesource.com/c/chromium/src/+/532294. However it doesn't seem to be getting much traction. I am reluctant to distro-patching as it's a rather large patch that often gets outdated with new chromium versions, with non-trivial conflicts to solve that sometimes involve re-architecting.

I am in the process of transitioning to snaps as the official way of getting chromium updates on Ubuntu. Once that's complete the maintenance burden will be largely alleviated and I might consider maintaining that patch.

[Michel-Ekimia (michel.ekimia)]

Hi Olivier.

I totally understand Canonical's position, Upstream is always better.

But we have here with google a competitive unfair distrortion.

They are doing their best to keep desktop linux behind by ignoring this patch for several years.

The result is that Linux laptops are not competitive in term of battery life and performance.

[Adam Smith (adamsmith)]

I've created a new bug to enable hardware acceleration via v4l2 codecs https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1799686 . Patches linked in the bug. This would give machines like the Raspberry Pi hardware acceleration (although currently the kernel needs patches too!).

[Robert Mader (robert.posteo)]

The dependency of this bug could probably changed from ogl-linux-beta to bug 1491303, as webrender will replace all the old rendering architecture and is already enabled on linux + intel in nightly.

[Robert Mader (robert.posteo)]

Although this bug is only about decoding support, I'd like to leave a note about the rendering: AFAIK rendering without any slow copies requires DMABUF support, so processes can share resources on the GPU.
For the Wayland backend this about to land, see bug 1552590, quote:

> Wayland dmabuf surface are located in GPU and can be attached as EGLImage or wl_buffer. It allows direct rendering to GPU and share the HW buffer across processes.

Also see bug 1010527

[Artyom Pozharov (artyom-pozharov)]

Yes, this bug affects me.

ср, 3 июл. 2019 г. в 05:54, Robert Mader <email address hidden>:

> Although this bug is only about decoding support, I'd like to leave a note
> about the rendering: AFAIK rendering without any slow copies requires
> DMABUF support, so processes can share resources on the GPU.
> For the Wayland backend this about to land, see bug 1552590, quote:
>
> > Wayland dmabuf surface are located in GPU and can be attached as
> EGLImage or wl_buffer. It allows direct rendering to GPU and share the
> HW buffer across processes.
>
> Also see bug 1010527
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1832816).
> https://bugs.launchpad.net/bugs/1424201
>
> Title:
> Web browsers lacking hardware-accelerated video decoding
>
> Status in Chromium Browser:
> Unknown
> Status in Mozilla Firefox:
> Confirmed
> Status in chromium-browser package in Ubuntu:
> Confirmed
> Status in firefox package in Ubuntu:
> Confirmed
> Status in chromium-browser package in CentOS:
> Unknown
>
> Bug description:
> The chromium team has done a great job to totally disable hardware
> decoding on Linux.
>
> Even ignoring the GPU blacklist does not enable GPU decoding.
>
> How ever the patch is quite simple and it's already working using this
> PPA ( using libVA ) :
>
> https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev
>
> the corresponding patch is here :
>
> http://bazaar.launchpad.net/~saiarcot895/chromium-browser/chromium-
> browser.trusty.beta/view/head:/debian/patches/enable_vaapi_on_linux.diff
>
> that would be great if we could have this patch indefault Ubuntu
> chromium build, this situation is really sad right now, google use
> linux to make chromeOS so chromebook are really good at playing videos
> but GNU/Linux chromium is slow at doing it.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/chromium-browser/+bug/1424201/+subscriptions
>

[Grmat (grmat)]

I've recently spent some time testing the implementations in various browsers. Chromium implements VA-API but it's disabled in official builds and fragile, breaking regularly. Webkit uses gstreamer-vaapi and I've had quality issues with that using both Intel (wrong colours) and AMD (blur) hardware.

I've seen Mozilla tracks issues for VA-API and VDPAU/NVDEC. Have you seen libplacebo[1]? It encapsulates the core video rendering used in mpv in a reusable library. mpv's video acceleration has been stable and flawless for both VAAPI and NVDEC (and DXVA, FWIW, but I've no experience with that) for years. In contrast to the browsers' and gstreamer va-api implementations, I've never had performance, stability or quality issues with mpv.

In fact, hundreds (thousands?) of Firefox users use several WebExtensions to have videos played in mpv[2],[3],[4].

I haven't worked with either Firefox or mpv code so I can only speak from a user's perspective right now. I just thought, it might be worth to consider implementing libplacebo instead of rolling an own implementation for va-api and vdpau/nvdec. libplacebo is licensed in LGPLv2.1.

Note: I initially wanted to post this in #1210726, quoting first comment:

> This bug will track support for hardware decoding on Linux and the various system available (VA-API, VDPAU, DXVA and others)
>
> There's so many frameworks available, it's hard to keep track!

But this one's locked. If it suits better there, feel free to move this comment.

[1]: https://github.com/haasn/libplacebo
[2]: https://addons.mozilla.org/en-US/firefox/addon/send-to-mpv-player/
[3]: https://addons.mozilla.org/en-US/firefox/addon/ff2mpv/
[4]: https://addons.mozilla.org/en-US/firefox/addon/play-with/

[Moz-m (moz-m)]

Also, please note that this would save the users a few GWh of electric energy and thus help protect the climate.

[Akarshanbiswas (akarshanbiswas)]

@grmat libplacebo is a renderer not a hardware decoder and is unfit to be used in a browser. First thing we need dmabuf support so that we can import decoded frames without copying them back to the system RAM and then display it on the screen. If I understand it correctly this is the toughest part.

[Robert Mader (robert.posteo)]

That should be bug 1572697 for Wayland.

[xtknight (xt-knight)]

If anyone's interested, I've attempted to add VP9 support to the VDPAU/VA-API wrapper for NVIDIA here: https://github.com/xtknight/vdpau-va-driver-vp9
Testers are welcome. I've used it successfully on 19.10 with chromium-vaapi.

[Michael Marley (mamarley)]

It looks like the VAAPI version is gone from the snap store again. Is there some reason why this hasn't been merged yet as mentioned in #96?

[Stransky (stransky)]

HW decoding support has landed at Bug 1616185.

*** This bug has been marked as a duplicate of bug 1616185 ***

[lilydjwg (lilydjwg)]

I'm glad to here that, but what about support for Firefox on X11? Is it given up?

[Stransky (stransky)]

(In reply to lilydjwg from comment #30)
> I'm glad to here that, but what about support for Firefox on X11? Is it given up?

Hardware decoding is platform independent so it generally works under X11 as it's implemented as Bug 1616185.
For whole playback chain see Bug 1610199 what's needed to be done fox X11.
I was closing this one as it refers to HW decode only, not the playback.
The overall video playback is tracked at Bug 1210726.

[Stransky (stransky)]

(In reply to lilydjwg from comment #30)
> I'm glad to here that, but what about support for Firefox on X11? Is it given up?

Filed Bug 1619523 for it.

[Jyavenard-9 (jyavenard-9)]

re-opening as a meta bug for VA-API implementation (including X11)

[Eric Riese (jebuswankel)]

Reminder that there is still a bounty for this issue at BountySource

https://www.bountysource.com/issues/55506502-add-va-api-hardware-decoding-support-on-linux

I got an email from BountySource that they're going to start taking old bounties for themselves starting *2020-07-1* (reproduced below)

In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.

```
Hi,
You are receiving this email because we are updating the Bountysource Terms of Service, effective 1st July 2020.

What's changing?
We have added a Time-Out clause to the Bounties section of the agreement:

2.13 Bounty Time-Out.
If no Solution is accepted within two years after a Bounty is posted, then the Bounty will be withdrawn and the amount posted for the Bounty will be retained by Bountysource. For Bounties posted before June 30, 2018, the Backer may redeploy their Bounty to a new Issue by contacting <email address hidden> before July 1, 2020. If the Backer does not redeploy their Bounty by the deadline, the Bounty will be withdrawn and the amount posted for the Bounty will be retained by Bountysource.

You can read the full Terms of Service here

What do I need to do?
If you agree to the new terms, you don't have to do anything.

If you have a bounty posted prior to June 30, 2018 that is not currently being solved, email us at <email address hidden> to redeploy your bounty. Or, if you do not agree with the new terms, please discontinue using Bountysource.

Thanks for reading

Bountysource Team
```

[Robert Mader (robert.posteo)]

(In reply to eric.riese from comment #34)
>
> In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.

That would certainly be Martin Stránský :)
Martin, do you want to claim the bounty? If you don't care about it / don't need it yourself, you could donate it or so.

[Stransky (stransky)]

(In reply to Robert Mader [:rmader] from comment #35)
> (In reply to eric.riese from comment #34)
> >
> > In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.
>
> That would certainly be Martin Stránský :)
> Martin, do you want to claim the bounty? If you don't care about it / don't need it yourself, you could donate it or so.

Okay, I'll try to claim it and send it to a charity.

[Stransky (stransky)]

This needs to be closed to claim the bounty at https://www.bountysource.com/issues/55506502-meta-add-va-api-hardware-decoding-support-on-linux so closing this one.

Wayland va-api decoding was implemented by Bug 1616185
X11 variant is pending at Bug 1619523

[Mathew Hodson (mhodson)]

This should depend on bug 1616185 and bug 1619523 so it's easier to find where the work happened.

[Daniel van Vugt (vanvugt)]

Tracking in bug 1947115 for the Firefox snap, although once fixed that's probably enough to close this one for Firefox too.

[Olivier Tilloy (osomon)]

Yes indeed.

[Michel-Ekimia (michel.ekimia)]

Could we track the firefox deb OOTB support ?

IMO FF deb still does not use the GPU OOTB to decode video.

[Olivier Tilloy (osomon)]

Indeed, before closing the firefox task for this bug, we need to look into the situation with the deb package.

[Daniel van Vugt (vanvugt)]

It looks like the old Chrome bug has been dead for a while:

  https://bugs.chromium.org/p/chromium/issues/detail?id=463440

But some newer bugs offer hope:

  https://bugs.chromium.org/p/chromium/issues/detail?id=1121948
  https://bugs.chromium.org/p/chromium/issues/detail?id=1262051

[Daniel van Vugt (vanvugt)]

Partial duplicate: bug 1816497

[Daniel van Vugt (vanvugt)]

Based on a recent report (https://discourse.ubuntu.com/t/problems-with-intel-iris-xe-gpu-hardware-video-acceleration-in-22-04/34849/4), I think Firefox is missing:

* Xwayland support: requires libva >= 2.17 in the gnome-42-2204 snap. That's also missing from jammy-updates.

* Intel 13th gen support: The fix for bug 2004237 needs to go in the gnome-42-2204 snap (patched iHD_drv_video.so).

[Michel-Ekimia (michel.ekimia)]

Yes this bug was supposed to solve it , But I don't think it did https://mzl.la/3W6JpMQ

[Daniel van Vugt (vanvugt)]

AV1 sounds like a third and orthogonal issue, separate from the two issues mentioned in comment #146.

The fact the bug is solved for the Ubuntu archive but not for snaps is a temporary problem. Longer term we will have graphics driver snaps being updated independently of app/gnome snaps. So then all apps will get the same level of hardware enablement at the same time.

[Michel-Ekimia (michel.ekimia)]

Vaapi in Flatpak Firefox works OOTB now ( Just a flag away) so definitely the best solution.

[Sebastien Bacher (seb128)]

It should also just be working with the firefox snap, what's your testcase?

[Daniel van Vugt (vanvugt)]

The firefox task is still open here, rightly or wrongly. Also the issue mentioned in comment #146 isn't resolved yet. Unless comment #146 is wrong?

EDIT: I just realised Firefox switching to native Wayland will avoid the first issue in comment #146. So unless someone is having trouble with Intel 13th gen still I can't think of a reason why it wouldn't work yet.

[Sebastien Bacher (seb128)]

@Daniel, #146 mentions xwayland but we switched firefox to use wayland by default since so it should work at least on waylands sessions? And #2 is about newer intel generation.

Said differently on a < 13th gen using wayland the videoacc should be working out of the box no?

[Daniel van Vugt (vanvugt)]

Yeah that's the same question I had in "EDIT" of comment #151. Someone just needs to test it on old and new hardware.

[Daniel van Vugt (vanvugt)]

Verified working in firefox (Wayland) on both Intel 12th and 13th gen systems. intel_gpu_top shows hardware video decoding is in use straight out of the box when viewing YouTube. The firefox snap versions tested were 122.0-2 and 122.0-2.1, both Noble systems.

Caveat: intel_gpu_top also shows very high values for Render/3D, so it's not power efficient. More work will need to be done toward zero-copy (or just less copy) in future.

[Daniel van Vugt (vanvugt)]

I only tested a 13th gen U chip in the above comment. It looks like 13th gen P chips should be supported so long as:

  intel-media-driver >= 22.3.1+dfsg1-1ubuntu2 (per bug 2004237)

which it looks like snaps are shipping with now.

Newer Intel 14th gen and Ultra chips might not be supported yet, but if so then please log a new bug about that.