Web browsers lacking hardware-accelerated video decoding

Bug #1424201 reported by Michel-Ekimia on 2015-02-21
92
This bug affects 18 people
Affects Status Importance Assigned to Milestone
Chromium Browser
Unknown
Unknown
Mozilla Firefox
Confirmed
Medium
chromium-browser (CentOS)
Unknown
Unknown
chromium-browser (Ubuntu)
Undecided
Olivier Tilloy
firefox (Ubuntu)
Undecided
Unassigned

Bug Description

The chromium team has done a great job to totally disable hardware decoding on Linux.

Even ignoring the GPU blacklist does not enable GPU decoding.

How ever the patch is quite simple and it's already working using this PPA ( using libVA ) :

https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev

the corresponding patch is here :

http://bazaar.launchpad.net/~saiarcot895/chromium-browser/chromium-browser.trusty.beta/view/head:/debian/patches/enable_vaapi_on_linux.diff

that would be great if we could have this patch indefault Ubuntu chromium build, this situation is really sad right now, google use linux to make chromeOS so chromebook are really good at playing videos but GNU/Linux chromium is slow at doing it.

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100401 SUSE/3.6.3-1.2 Firefox/3.6.3
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100401 SUSE/3.6.3-1.2 Firefox/3.6.3

There should be a feature that enables HTML5 video to use GPU acceleration so this will take stress off the CPU. It should work on Ubuntu, OpenSuse, Fedora, etc, and it should work with all graphics cards including onboard video.

Reproducible: Always

This may depend on or be a dupe of bug 556027

This is filed before, and even partially implemented (see bug 555839 for instance), but there's still lots of work to do. Especially on the cross-platform behaviour (most work is done in the windows side at the moment).

13 comments hidden view all 124 comments
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in chromium-browser (Ubuntu):
status: New → Confirmed
Michel-Ekimia (michel.ekimia) wrote :

Hi chad , For few versions chromium was unable to use the vaapi , It's now possible again using the patch from the dev branch :

https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev

Can you integrate this patch ?

Also a lot of GPU VP8/VP9 decoding become possible with nvidia and Intel cards so that will be interresting to test.

95 comments hidden view all 124 comments

This would do decoding only ; not rendering (yet)

Hardware decoding on Intel Ironlake should be used only for 720p and lower.
Hardware 1080p decoding on Intel Ironlake is slower than software decoding on CPU: https://bugs.freedesktop.org/show_bug.cgi?id=47858

GPU model from about:support
Intel Open Source Technology Center -- Mesa DRI Intel(R) Ironlake Mobile

I need to fill separate issue about VA-API on Ironlake or drop this info here is fine?

(In reply to russianneuromancer from comment #1)
> Hardware decoding on Intel Ironlake should be used only for 720p and lower.
> Hardware 1080p decoding on Intel Ironlake is slower than software decoding
> on CPU: https://bugs.freedesktop.org/show_bug.cgi?id=47858
>
> GPU model from about:support
> Intel Open Source Technology Center -- Mesa DRI Intel(R) Ironlake Mobile
>
> I need to fill separate issue about VA-API on Ironlake or drop this info
> here is fine?

We have a generic preference to disable hardware acceleration, I'll hook it up to vaapi, but I have no intention at this stage to make specific cases for particular chipset.

Would have to be in a separate bug.

83 comments hidden view all 124 comments

Hello,

I was surprised to see that GStreamer is no longer needed for video playback and that only FFmpeg/Libav is needed now:

https://bugzilla.mozilla.org/show_bug.cgi?id=1207429

Which is very nice.

But I have a question:

There's a bug regarding hardware accelerated video playback with GStreamer, see:

https://bugzilla.mozilla.org/show_bug.cgi?id=894372

So, I was hoping that Firefox would support fully hardware accelerated video playback now that GStreamer is no longer required and went ahead and installed the latest Firefox Nightly 44.0a1 on Ubuntu 15.04 to test it.

Video playback indeed was working without GStreamer, but the video playback still didn't seem to be fully hardware accelerated ;(.

Why is that?

Why is the video playback on Linux still not fully hardware accelerated :(?

Regards

I'm on Fedora 23, ffmpeg and vaapi libs installed. Intel Sandy-Bridge hd3000. HW decoding with libva works in native apps.

Testing FF 45 nightly , in about:support I see : "Supports Hardware H264 Decoding No;"

Is there a way to test that ffmpeg is in fact used ? And for for HW accel?

Jean-Yves, can you help answer?
We could also maybe document this on SUMO.

There is no unique nor "official" software path to have hardware acceleration on linux.
Every chip makers have designed their own framework to do so; none of them compatible with one another.

You may want to track bug 1210729 or bug 1210727.

The only GStreamer plugin allowing hardware acceleration decoding was a VAAPI plugin: it was buggy and extremely unstable nor did it provide any significant speed improvement.

You're much better off using ffmpeg

As far as I know, there is 2 main distinct framework for hardware acceleration on linux - VDPAU and VAAPI. Eg. VLC or Mplayer supports this, but not Firefox.

I read some comments, that GStreamer + VAAPI plugin works for someone in Firefox, but personally I could never manage it to work (video plays, but not HW accelerated), move to ffmpeg seems to be a good decision.

As far as I know, VLC also uses ffmpeg internally and is able to use HW acceleration frameworks, so I suppose there should be a way for Firefox to use it also.
Now video playback in Firefox on Linux means quite high CPU usage (for me - fullHD H.264 HTML5 video on recent Broadwell i7 CPU takes ~130% of CPU - more than entire one CPU core).

(In reply to Jean-Yves Avenard [:jya] from comment #7)
> There is no unique nor "official" software path to have hardware
> acceleration on linux.
> Every chip makers have designed their own framework to do so; none of them
> compatible with one another.

That isn't really true, or at best several years out of date. There are effectively only two frameworks, VDPAU (supported by NVidia's proprietary drivers and adopted by the open source NVidia and AMD drivers) and VAAPI (for Intel). Each can use the other as a back-end if necessary, but as far as I know you could be right that it doesn't work well. AMD's proprietary driver supposedly has bindings for something else, but they never provided any support or documentation, so that can be ignored.

> You may want to track bug 1210729 or bug 1210727.
>
> The only GStreamer plugin allowing hardware acceleration decoding was a
> VAAPI plugin: it was buggy and extremely unstable nor did it provide any
> significant speed improvement.

VAAPI is a little behind VDPAU in terms of maturity and take-up, but both now work well in every other significant Linux video player. I heard the reason it was slow in Firefox was because they made it render into main RAM and copied the data to the framebuffer entirely in software. Use VAAPI properly and the CPU usage barely registers.

> You're much better off using ffmpeg

Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy for Firefox to use that route to gain GPU acceleration.

For what it's worth, embedded systems / SoC vendors often provide support for hardware-accelerated video decoding via OpenMax (omx), in addition to any vendor-specific APIs that may also exist.

(In reply to Tony Houghton from comment #9)
> > You're much better off using ffmpeg
>
> Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy
> for Firefox to use that route to gain GPU acceleration.

This is irrelevant to what ffmpeg does, nor if it was compiled to support either vaapi or vdpau. Compiling FFmpeg with vdpau or vaapi support won't automagically make applications using ffmpeg start using hardware acceleration unfortunately.

The P in VDPAU stands for Presentation. The only way you can paint an image decoded with VDPAU is to *P*resent it directly using VDPAU. It can't be rendered the way all the other images are typically painted.
VAAPI is similar (There's also the issue that an entire generation of intel GPUs are buggy)

We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have no other options but copying the decoded image out of the GPU memory into a software surface (which is exactly what the vaapi gstreamer plugin does).

Doing so almost always annihilate any benefits, as copying the image back from the GPU is typically slow (and even more so with nvidia devices)

To make things worse, at this stage, firefox on linux doesn't have hardware accelerated compositor either (OpenGL isn't enabled by default).

We won't be using FFmpeg's hwaccel layer to access vaapi or vdpau (it's only a helper)

(BTW, AMD's own XvBA is perfectly documented)

(In reply to t.i.m from comment #10)
> For what it's worth, embedded systems / SoC vendors often provide support
> for hardware-accelerated video decoding via OpenMax (omx), in addition to
> any vendor-specific APIs that may also exist.

Yes. We are currently developer an OpenMax decoder (bug 1224887). This is for our own FirefoxOS devices, and I'm not sure how easy it will be to enable on other platforms, but the majority of the work will be there.

(In reply to Jean-Yves Avenard [:jya] from comment #11)
> (In reply to Tony Houghton from comment #9)
> VAAPI is similar (There's also the issue that an entire generation of intel GPUs are buggy)

Maybe I haven't noticed this issue, but I can say, that I didn't noticed problems on Ivy Bridge, Haswell and Broadwell GPUs (VLC with VAAPI works just fine).

> We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have
> no other options but copying the decoded image out of the GPU memory into a
> software surface (which is exactly what the vaapi gstreamer plugin does).
>
> Doing so almost always annihilate any benefits, as copying the image back
> from the GPU is typically slow (and even more so with nvidia devices)

How is the situation different on Windows? Just trying to get, why is it such a problem.

> To make things worse, at this stage, firefox on linux doesn't have hardware
> accelerated compositor either (OpenGL isn't enabled by default).

OpenGL on linux can be enabled manually, I'm using it for months without apparent problems.
GPU Accelerated Windows: 1/1 OpenGL (OMTC) (in about:support)

If OpenGL compositor on Linux will be working without problems, would be still needed to copy image from GPU to software, or can it be avoided? If I remember right, I read some article about gstreamer+vaapi in Firefox and it stated, that this copying is killing VAAPI performance.

I'm just trying to figure out, what is needed to get VAAPI/VDPAU working...

(In reply to martin from comment #13)

> > Doing so almost always annihilate any benefits, as copying the image back
> > from the GPU is typically slow (and even more so with nvidia devices)
>
> How is the situation different on Windows? Just trying to get, why is it
> such a problem.

On windows we go through the Windows Media Foundation framework for which the intel driver provides a hardware decoder and plug in automatically to WMF.

So the decoder returns a DXVA surface, which we can render immediately on the screen as we have a DXVA and D3D compositor.

On linux, there's no such common framework.

> OpenGL on linux can be enabled manually, I'm using it for months without
> apparent problems.
> GPU Accelerated Windows: 1/1 OpenGL (OMTC) (in about:support)
>
> If OpenGL compositor on Linux will be working without problems, would be
> still needed to copy image from GPU to software, or can it be avoided?

VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then be drawn without having to first copy it to software.
Though, IIRC, Intel was talking about deprecating that API so that you must go through the VAAPI framework to draw the surface. If they did that, we would have to do the same work as we need for VDPAU

> If I remember right, I read some article about gstreamer+vaapi in Firefox and it
> stated, that this copying is killing VAAPI performance.
this is what I stated earlier on !

>
> I'm just trying to figure out, what is needed to get VAAPI/VDPAU working...

So for VDPAU, we need a VDPAU compositor. Something that would allow us to draw a VDPAU surface into the screen.
The way VDPAU works typically is that you create an OpenGL surface, assign it a colorkey and then tell VDPAU to draw on that color key. That allows to draw the VDPAU surface into the OpenGL surface and then render that surface on the screen using OpenGL.
It's similar to your blue screen stuff used on TV and movies.

Those are all things that need to be done if we want fully accelerated hardware decoding. Where we use the GPU to decode a video frame, and stay within the GPU to render it on the screen.
Right now, we can't.

(In reply to Jean-Yves Avenard [:jya] from comment #11)
> (In reply to Tony Houghton from comment #9)
> > > You're much better off using ffmpeg
> >
> > Only in versions of ffmpeg that support VDPAU and VAAPI. I'd be very happy
> > for Firefox to use that route to gain GPU acceleration.
>
> This is irrelevant to what ffmpeg does, nor if it was compiled to support
> either vaapi or vdpau. Compiling FFmpeg with vdpau or vaapi support won't
> automagically make applications using ffmpeg start using hardware
> acceleration unfortunately.

I know that, but doesn't ffmpeg provide some sort of API wrapper so that if you're already using it for software decoding it's easier to adapt your code for acceleration instead of starting again from scratch? And even if not, its demuxers and audio decoders should still be useful alongside VDPAU/VAAPI.

> The P in VDPAU stands for Presentation. The only way you can paint an image
> decoded with VDPAU is to *P*resent it directly using VDPAU. It can't be
> rendered the way all the other images are typically painted.
> VAAPI is similar (There's also the issue that an entire generation of intel
> GPUs are buggy)

That's just one type of GPU, there are many others that could be supported without that sort of hardware/driver problem.

> We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have
> no other options but copying the decoded image out of the GPU memory into a
> software surface (which is exactly what the vaapi gstreamer plugin does).
>
> Doing so almost always annihilate any benefits, as copying the image back
> from the GPU is typically slow (and even more so with nvidia devices)
>
> To make things worse, at this stage, firefox on linux doesn't have hardware
> accelerated compositor either (OpenGL isn't enabled by default).

But OpenGL can be enabled, and Firefox's support for WebGL on Linux seems quite good. Seeing as you can use an accelerated surface for WebGL canvases, is it really that much harder to set one up for video elements so you can use VDPAU or VAAPI effectively?

I really can't agree that we're better off with software-only ffmpeg and that you shouldn't make every effort with GPU decoding. I've experienced that even some very recent generation desktop CPUs (eg Core i3 @ 3GHz) can't maintain the proper frame rate with some HD video. And 4K videos are already appearing on youtube. (Although we'd presumably have to wait for Firefox's DASH support for these).

> We won't be using FFmpeg's hwaccel layer to access vaapi or vdpau (it's only
> a helper)
>
> (BTW, AMD's own XvBA is perfectly documented)

Nevertheless, there seems to be something blocking its adoption. But I think there was talk of exposing it via VAAPI if and when it does get supported properly, anyway.

Moreover hardware accelerated decoding seems to fit with project candle (rather than software decoding)

(In reply to Tony Houghton from comment #15)

> I really can't agree that we're better off with software-only ffmpeg and
> that you shouldn't make every effort with GPU decoding. I've experienced

What I said was that you're better of using ffmpeg over gstreamer with the vaapi plugin
Memory bandwidth with 4K quickly becomes the bottleneck. On windows in a current work in progress simply removing a single frame copy sped up the decoding frame rate by 6%; this is huge.

At not time did I state that we didn't want to do full GPU acceleration.

Just that things aren't trivial as what you appear to think. Why do you think chrome dropped it entirely very recently?
If it was that easy and worked well, it would be done already.

(In reply to Jean-Yves Avenard [:jya] from comment #17)
> (In reply to Tony Houghton from comment #15)
>
> > I really can't agree that we're better off with software-only ffmpeg and
> > that you shouldn't make every effort with GPU decoding. I've experienced
>
> What I said was that you're better of using ffmpeg over gstreamer with the
> vaapi plugin
> Memory bandwidth with 4K quickly becomes the bottleneck. On windows in a
> current work in progress simply removing a single frame copy sped up the
> decoding frame rate by 6%; this is huge.
>
> At not time did I state that we didn't want to do full GPU acceleration.

OK. I just got the impression you weren't interested in supporting it. I hope you do manage to get it working.

> Just that things aren't trivial as what you appear to think. Why do you
> think chrome dropped it entirely very recently?

Have they dropped it altogether, even for ChromeOS, now? I know they disabled it for Linux, but didn't remove it before, and Chromium could be recompiled with it enabled with a few changes to some #if statements etc. True, it has been very unreliable, but in the sense that it never works in some versions/systems and falls back to software rather than in the sense that it keeps crashing; when it does work it's stable enough. But the reason they gave for disabling it in Linux was that they had nobody to work on it, not that they couldn't get it working.

> If it was that easy and worked well, it would be done already.

It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to support it together with all the other things a browser has to do makes it harder, but surely not bordering on impossible, and I'm sure at Mozilla you have the programming talent to make it work.

(In reply to Tony Houghton from comment #18)

> It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to

well, they have a single window to worry about. I got it working just fine in mythtv too (as well as vdpau).
They don't have to worry about compositing with any other objects, images or handle someone who has decided to make the video element move around in an html page.

(In reply to Jean-Yves Avenard [:jya] from comment #14)
> (In reply to martin from comment #13)

>
> VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then
> be drawn without having to first copy it to software.
> Though, IIRC, Intel was talking about deprecating that API so that you must
> go through the VAAPI framework to draw the surface. If they did that, we
> would have to do the same work as we need for VDPAU

If you are talking about this api:http://cgit.freedesktop.org/vaapi/libva/tree/va/egl/va_egl.h,
Then yes, it is not implemented in driver.

But there are other ways to map deocoded va sufaces to eglimage through dmabuf and vaAcquireBufferHandle.

A detailed implementation with comments here: https://github.com/01org/gstreamer-vaapi/blob/master/gst-libs/gst/vaapi/gstvaapisurface_egl.c

Anther implementation for the same here: https://github.com/01org/libyami/blob/master/egl/egl_vaapi_image.cpp

(In reply to Jean-Yves Avenard [:jya] from comment #14)
> (In reply to martin from comment #13)
> VAAPI has an API to map a VAAPI surface to an OpenGL surface, it could then
> be drawn without having to first copy it to software.
> Though, IIRC, Intel was talking about deprecating that API so that you must
> go through the VAAPI framework to draw the surface. If they did that, we
> would have to do the same work as we need for VDPAU

> So for VDPAU, we need a VDPAU compositor. Something that would allow us to
> draw a VDPAU surface into the screen.
> The way VDPAU works typically is that you create an OpenGL surface, assign
> it a colorkey and then tell VDPAU to draw on that color key. That allows to
> draw the VDPAU surface into the OpenGL surface and then render that surface
> on the screen using OpenGL.
> It's similar to your blue screen stuff used on TV and movies.

Ok, thank you for explaining.
So, if I understood correctly, the blocker now is the OpenGL compositor, which is now disabled by default on linux. I guess there are some reasons for that, some issues?
When OpenGL compositor will be working fine, then it could be possible to use VAAPI (using that - maybe in future deprecated - API to map VAAPI surface to OpenGL surface) - that would be the (relatively) easy way, right?

In fact I'm not sure, what VDPAU/VAAPI compositor means - does it mean entire new compositor (like the OpenGL one, which probably still has some issues (guessing from it's not enabled by default)), or is it some "overlay" on OpenGL compositor?
The "entirely new" option would be bad, I'm afraid it would take very long time... It's really a problem on linux, I also experienced lags and dropped frames in some HD videos, so I hope it wouldn't take years to support video HW acceleration.

(In reply to Jean-Yves Avenard [:jya] from comment #19)
> (In reply to Tony Houghton from comment #18)
>
> > It works well in mpv, mplayer, vlc, kodi etc etc. I realise having to
>
> well, they have a single window to worry about. I got it working just fine
> in mythtv too (as well as vdpau).
> They don't have to worry about compositing with any other objects, images or

Actually they do, most of those other players can overlay GUIs etc on the video.

> handle someone who has decided to make the video element move around in an
> html page.

Ick. But can't they also do that with canvases, so you would already have come up with some solution for WebGL?

(In reply to Tony Houghton from comment #22)
> (In reply to Jean-Yves Avenard [:jya] from comment #19)
> > handle someone who has decided to make the video element move around in an
> > html page.
>
> Ick. But can't they also do that with canvases, so you would already have
> come up with some solution for WebGL?

Think about scrolling a video on a touch screen. The video needs to scroll coherently with the rest of the image as well as be able to support pinch-zoom. This becomes more difficult if the video is presented through a parallel presentation path as it would be with VDPAU. Ultimately the accelerated decoding on Linux doesn't match with Firefox's compositor model. In order to support accelerated decoding on Linux, someone needs to write some code to bridge that gap. It is not simply a matter of enabling acceleration.

> > They don't have to worry about compositing with any other objects, images or
>
> Actually they do, most of those other players can overlay GUIs etc on the
> video.

You are ignoring the simple fact that video players are architecturally simpler than browsers. Video players have no other purpose than playing video.

We are definitely interested in getting hardware acceleration to work on Linux. If it is as easy as you seem to think it is and you've got some spare time on your hands then it might be a fun project for you.

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #23)
> (In reply to Tony Houghton from comment #22)
> >
> > Ick. But can't they also do that with canvases, so you would already have
> > come up with some solution for WebGL?
>
> Think about scrolling a video on a touch screen. The video needs to scroll
> coherently with the rest of the image as well as be able to support
> pinch-zoom. This becomes more difficult if the video is presented through a
> parallel presentation path as it would be with VDPAU. Ultimately the
> accelerated decoding on Linux doesn't match with Firefox's compositor model.
> In order to support accelerated decoding on Linux, someone needs to write
> some code to bridge that gap. It is not simply a matter of enabling
> acceleration.

I know, but what I was saying is that you already have a way of dealing with specialised rendering surfaces for WebGL canvases. What's so different about VDPAU and VAAPI surfaces? Or does Firefox's WebGL implementation have the same issue that the gstreamer-vaapi implementation did with inefficient copying from an off-screen GPU buffer to the window surface?

> > > They don't have to worry about compositing with any other objects, images or
> >
> > Actually they do, most of those other players can overlay GUIs etc on the
> > video.
>
> You are ignoring the simple fact that video players are architecturally
> simpler than browsers. Video players have no other purpose than playing
> video.

They may not have to deal with scrolling and clipping, but some of them can do a lot more. Kodi has a very sophisticated OSD, supporting non-rectangular transparency and arbitrary downloaded images. It can run full-screen or in a window and either play the video taking up the entire window/screen, or just part of it.

But I take your point, it would have been designed as a video player that has to fit static elements over and around the video, which is back-to-front compare to a browser.

> We are definitely interested in getting hardware acceleration to work on
> Linux.

Thank you for making an effort. I got the wrong impression to start with, because I'm more used to the GNOME bugzilla, where developers pointing out difficulties is usually their making excuses to ignore important issues for years. Ironically though, their browser does seem to support VAAPI at least, but of course it's a long way behind Firefox in other areas.

> If it is as easy as you seem to think it is and you've got some spare
> time on your hands then it might be a fun project for you.

It wouldn't be an efficient use of developer time. I don't mean that I've got better things to do, but that you probably have better people for the job. I don't already know how to use VDPAU and/or VAAPI anyway, but even if I did, I think it would be easier for Firefox developers to learn VDPAU/VAAPI than a VDPAU/VAAPI expert to learn their way around Firefox's code base.

i am new in the bugzilla, hello to all the world. i want to say that with archlinux all plugins installed, firefox nightly for testing too, nothing have accelerated video rendering(and my intel integradted is capable for that) I know that you, all the develops have much work.. but.. it seems that windows have better hardware capabilities, and not only in this things is better, in the html5 support too I think, boys.. please.. in Linux the predetermined browser is firefox while in windows google chrome is the most used, chrome/chromium develops aren't interested in hardware acceleration either for linux..I think is the time for the change..not only for this request, but for all request of linux that windows have implemented now

and don't tell me the drivers isn't good enoug because intel drivers are very well, amd open source too, and nvidia propietary drivers too, create a black list with amd cloused drivers and nvidia open drivers andfixed for the stablity

I encourage everyone here. It is the only remaining flaw in Linux

Aparently there is low to none interest from devs about this issue, it's been almost 3 months without any response.

Meanwhile I tried Chromium, that can support VAAPI (well not oficially, but VAAPI support is there, it has to be patched to work not only in Chrome OS). Since Chromium devs aren't interested in supporting linux, it's not really straightforward to get it to work, but I've made it and HW acceleration is working (CPU usage drops in video playback). I tried it only to be sure it's working elsewhere (and that it could be done in Firefox too).

VAAPI also works well in normal video players like VLC (on Intel Broadwell IGPU), so I don't think buggy drivers is valid reason for not implementing that. There could be configuration option to turn HW accel off, if on some machine it's causing a problem.

It's a shame that Firefox devs don't care much about Linux too. Firefox is imo still most used browser in Linux and video playback with Firefox is really a pain.
It's really frustrating that even with new Intel Broadwell i7, playback of 1080p video causes high CPU usage (thus low life time on battery) and that the video is still not fluent, it still shutters/lags.

I don't know what else to say. I think it's one of most serious issues of Firefox in Linux now.
Are there any developers interested in this, are there any plans about fixing this issue sometime in near future?

36 comments hidden view all 124 comments

Seems like this issue also affects Opera users.

Saikrishna Arcot clarified that this feature request doesn't affect Opera users.

96 comments hidden view all 124 comments

Mass change P2 -> P3

Is there a reason the priority would go from P2 to P3?

Lack of hardware acceleration is the worse offender on Linux right now. Some youtube videos can not even play at a decent framerate because of that, especially the 4K / 360 ones. And that's without speaking of the heat generated by those poor CPUs...

I do not get why it would not even go the other way, right to P1 ?

linux had have the firefox browser by default in all distros I think much users of firefox are from linux and the only thing we get from mozilla a long waiting for a feature that is very needed.. between google with chromium and the denied featured requests and firefox not to working on features for firefox, linux in browser category is very late and the battery for laptops is damaged with linux and video viewing with browsers, I feel really annoying

I fully agree, video acceleration is a must have feature now a days. Please solve it! Don't force me to move to Chrome....

I still have hope too to see full VDPAU/VAAVI support in order to have hardware video acceleration.

Gentle reminder:

* Our guidelines ask you don't argue about priority setting on bugs. Triage leads and Mozillians working on the bug set the priority. See: https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
* "Me too" or "Fix this or I'm moving to Chrome/Lynx/Gopher/Edge/Webkit" comments are off topic and just add noise.

description: updated
101 comments hidden view all 124 comments
dino99 (9d9) wrote :

hi Michel
i understand the general frustation, but chromium is not the only choice after all. Its clear that nothing will be done to support that feature on linux (pure management resources decision)

102 comments hidden view all 124 comments

Hi,

in firefox release/51 and beta/52 in about:support one can see "Hardware H264 Decoding" status.

However in firefox aurora/53 and nigthly/54, this line is no longer visible.

Was this done on purpose or is this an omission?

BTW, how about adding a line about "Hardware VP9 Decoding" since recent Intel GPU do support hardware VP9 decoding?

(In reply to Laurent Bonnaud from comment #10)
> in firefox release/51 and beta/52 in about:support one can see "Hardware
> H264 Decoding" status.
>
> However in firefox aurora/53 and nigthly/54, this line is no longer visible.
>
> Was this done on purpose or is this an omission?
>
> BTW, how about adding a line about "Hardware VP9 Decoding" since recent
> Intel GPU do support hardware VP9 decoding?

Laurent, if this is still reproducible – I only have 45.9.0 here –, could you please create a separate report for this issue? That’d be great.

@ehumphries, if I interpret the bug comments correctly, you flagged this report with *NEEDINFO*. Could you please tell us, what information is needed to get the feature implemented?

"needinfo" is intended to call the attention of a specific person to the bug - I need information from X, to answer question Y.

It's not intended as a nonspecific "we need more information to figure this out" thing, but it's true the term is a bit opaque.

104 comments hidden view all 124 comments
Olivier Tilloy (osomon) wrote :

Tentatively assigned myself to re-assess the situation, and try to understand if enabling VAAPI support is doable with a distro-patch, and how much work that would represent.

Changed in chromium-browser (Ubuntu):
assignee: nobody → Olivier Tilloy (osomon)
105 comments hidden view all 124 comments

(In reply to Paul Menzel from comment #11)
> Laurent, if this is still reproducible – I only have 45.9.0 here –, could
> you please create a separate report for this issue? That’d be great.

Is there already a separate report for this issue? There are also some other useful informations gone (i can't remember exactly, but e.g. regarding OpenGL acceleration).

Yes, we also need accelerated layers to be able to render the frames. And right now, this has taken a set back and the feature has been disabled for now.

(In reply to Jean-Yves Avenard [:jya] from comment #15)
> Yes, we also need accelerated layers to be able to render the frames. And
> right now, this has taken a set back and the feature has been disabled for
> now.

Have you started doing work on this and you wait for accelerated layers to land in order to merge it or you will start after the layers?

Just FYI: https://chromium-review.googlesource.com/c/532294 this will be [chrome-parity] soon

"The Quantum Flow (QF) initiative is a cross-component effort to improve user perceived responsiveness and performance."
Could you please triage this if it would meet your demands for 57 or 58?

Video hardware decoding is very desirable for Linux devices that:
* do not have a powerful CPU
* should be energy-efficient (laptops)
* want to fluently play 4K videos on YouTube while using the CPU for compiling Nightly. ;-)

There is new code from Chromium in comment 17 providing VAAPI hardware video decoding that could and should be shared, or not?
If Mozilla and Google both would fix bugs it would result in far better stability for all users. Would it be possible?
At least we have example code now.

(In reply to Jean-Yves Avenard [:jya] from comment #15)
> Yes, we also need accelerated layers to be able to render the frames.

Could we say we have a dependency to WebRender (instead of the current painting code) here, to prevent double work?

Personally, I do NOT care about accelerated layers for rendering frames. Who cares?
I just want to use "accelerated video decoding via HW" because solving the video codec exhaust many CPUs, not rendering frames after. Yes rendering via OGL will probably give much better results but who cares? We need accelerated youtube videos ASAP.

Also, read the first reply of :jya at the topic,
"for just decoding, not rendering (yet)."

I believe, after 2 years, devs need to complete first step using current painting stack. Because this gonna hurt firefox project much more than other bugs. Since linux users cannot watch simple videos with FF, they gonna switch to Chrome soon.

110 comments hidden view all 124 comments
Michel-Ekimia (michel.ekimia) wrote :

@dino99 : are you working at google on chromium ?

Your argument is pretty wrong, there is Zero impact if this feature is hidden with a flag that one could enable when deployed.

Today we are stuck because we need to recompile.

Sorry but one could think that google wants to slow DesktopLinux progress.

111 comments hidden view all 124 comments

Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)

111 comments hidden view all 124 comments
Michel-Ekimia (michel.ekimia) wrote :

Hi olivier , can we see if the Ubuntu package can include the patch at least for 18.04 ? We talk about version 66 which is available now.

Olivier Tilloy (osomon) wrote :

The current upstream effort is tracked at https://chromium-review.googlesource.com/c/chromium/src/+/532294. However it doesn't seem to be getting much traction. I am reluctant to distro-patching as it's a rather large patch that often gets outdated with new chromium versions, with non-trivial conflicts to solve that sometimes involve re-architecting.

I am in the process of transitioning to snaps as the official way of getting chromium updates on Ubuntu. Once that's complete the maintenance burden will be largely alleviated and I might consider maintaining that patch.

Michel-Ekimia (michel.ekimia) wrote :

Hi Olivier.

I totally understand Canonical's position, Upstream is always better.

But we have here with google a competitive unfair distrortion.

They are doing their best to keep desktop linux behind by ignoring this patch for several years.

The result is that Linux laptops are not competitive in term of battery life and performance.

summary: - Chromium does not use the GPU to decode videos
+ Web browsers lacking hardware-accelerated video decoding
Changed in firefox (Ubuntu):
status: New → Confirmed
Adam Smith (adamsmith) wrote :

I've created a new bug to enable hardware acceleration via v4l2 codecs https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1799686 . Patches linked in the bug. This would give machines like the Raspberry Pi hardware acceleration (although currently the kernel needs patches too!).

73 comments hidden view all 124 comments

(In reply to Jean-Yves Avenard [:jya] from comment #11)
> We do not have a VDPAU compositor (nor a OpenGL-VAAPI), so we currently have
> no other options but copying the decoded image out of the GPU memory into a
> software surface (which is exactly what the vaapi gstreamer plugin does).
>
> Doing so almost always annihilate any benefits, as copying the image back
> from the GPU is typically slow (and even more so with nvidia devices)
>
> To make things worse, at this stage, firefox on linux doesn't have hardware
> accelerated compositor either (OpenGL isn't enabled by default).
>
> We won't be using FFmpeg's hwaccel layer to access vaapi or vdpau (it's only
> a helper)

Now that WebRender is shipped in Firefox, is there some way to effectively use VAAPI for video? VDPAU seems dead upstream, so not a good option.

Note there is an ongoing work to enable VAAPI in Chromium as well for Linux (and not just ChromeOS). For example: https://github.com/chromium/chromium/commit/31225b9c5f3f685d65f742dc129241c30c32157c.

Changed in firefox:
importance: Unknown → Wishlist
status: Unknown → New

One of the use cases where this is sorely needed are WebRTC video conferencing applications like WebEx and the like. Without GPU acceleration, they become very CPU heavy, which cripples them on laptops especially.

Chris Cheney (ccheney) wrote :

As neither Chrome/Chromium or Firefox has any real interest in Linux support can we get Ubuntu to apply the patch? The patch has existed for 5 years already.

Even a partial solution where its built in but you still have to enable it in config would be fine. Having to hunt down a ppa to get working video in 2019 on Linux is absurd.

Otherwise users would be much better off moving to another distro that supports it such as Arch, Fedora, etc as noted above.

Chris Cheney (ccheney) wrote :

Fedora started enabling video acceleration in this build late last year:

* Tue Nov 27 2018 Tom Callaway <email address hidden> - 70.0.3538.110-2
- enable vaapi support (thanks to Akarshan Biswas for doing the hard work here)

Saikrishna Arcot (saiarcot895) wrote :

There are snap packages in Ubuntu that enable hardware video acceleration.

Good to know. What is downside of enabling hardware acceleration in deb packages too?

Daniel van Vugt (vanvugt) wrote :

Last I heard, Ubuntu "applying the patch" wasn't feasible because it kept breaking due to upstream changes in Chromium. It's not a patch we can carry without someone volunteering to maintain it. But oSoMoN is the expert here and will have more current information...

P.S. You can watch YouTube videos with hardware acceleration (and no need for patches) using these instructions: https://wiki.ubuntu.com/IntelQuickSyncVideo#YouTube

Olivier Tilloy (osomon) wrote :

As stated by Daniel, the problem is one of maintenance. Maintaining a patch against a large and fast-moving code base such as chromium has a cost, which is why it's always preferable to have such changes upstreamed.

As it is clear that upstream is not going to enable hardware-accelerated video decoding on Linux anytime soon (if ever), the distro patch is indeed the way to go.

That fedora patch is relatively small and maintainable as it is today, so I'll update the snap package to include it (until now it was in a dedicated channel).

In response to comment #92, there is a plan to deprecate the chromium-browser deb package in favour of the snap in the near future, so development and testing focus is on the snap.

Chris Cheney (ccheney) wrote :

wrt #91 which snaps might those be?

If its the regular 'chromium' snap it certainly doesn't make any mention of the fact it supports hardware video acceleration or includes the vaapi patch.

# snap info chromium
name: chromium
summary: Chromium web browser, open-source version of Chrome
publisher: Canonical✓
contact: https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bugs?field.tag=snap
license: Apache-2.0 AND BSD-3-Clause AND LGPL-2.0 AND LGPL-2.1 AND MIT AND MS-PL AND (GPL-2.0+ OR LGPL-2.1+ OR MPL-1.1)
description: |
  An open-source browser project that aims to build a safer, faster, and more stable way for all
  Internet users to experience the web.
snap-id: XKEcBqPM06H1Z7zGOdG5fbICuf8NWK5R
channels:
  stable: 73.0.3683.75 2019-03-14 (660) 161MB -
  candidate: 73.0.3683.75 2019-03-13 (660) 161MB -
  beta: 73.0.3683.75 2019-03-13 (660) 161MB -
  edge: 74.0.3729.22 2019-03-20 (667) 162MB -

Olivier Tilloy (osomon) wrote :

The vaapi-enabled snap currently lives in the candidate/vaapi channel, which was automatically closed due to lack of updates, but an update bringing it to the latest stable version is pending publication, so it will soon be re-opened and available.

Once confirmed this works as expected, I plan to merge this branch into the stable one, at which point a separate channel won't be required.

Changed in firefox:
importance: Wishlist → Medium
24 comments hidden view all 124 comments

The dependency of this bug could probably changed from ogl-linux-beta to bug 1491303, as webrender will replace all the old rendering architecture and is already enabled on linux + intel in nightly.

23 comments hidden view all 124 comments

*** This bug has been marked as a duplicate of bug 1210726 ***

Changed in firefox:
status: New → Invalid
Changed in firefox:
importance: Medium → Unknown
status: Invalid → Unknown
Changed in firefox:
importance: Unknown → Medium
status: Unknown → Confirmed
24 comments hidden view all 124 comments

Although this bug is only about decoding support, I'd like to leave a note about the rendering: AFAIK rendering without any slow copies requires DMABUF support, so processes can share resources on the GPU.
For the Wayland backend this about to land, see bug 1552590, quote:

> Wayland dmabuf surface are located in GPU and can be attached as EGLImage or wl_buffer. It allows direct rendering to GPU and share the HW buffer across processes.

Also see bug 1010527

Yes, this bug affects me.

ср, 3 июл. 2019 г. в 05:54, Robert Mader <email address hidden>:

> Although this bug is only about decoding support, I'd like to leave a note
> about the rendering: AFAIK rendering without any slow copies requires
> DMABUF support, so processes can share resources on the GPU.
> For the Wayland backend this about to land, see bug 1552590, quote:
>
> > Wayland dmabuf surface are located in GPU and can be attached as
> EGLImage or wl_buffer. It allows direct rendering to GPU and share the
> HW buffer across processes.
>
> Also see bug 1010527
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1832816).
> https://bugs.launchpad.net/bugs/1424201
>
> Title:
> Web browsers lacking hardware-accelerated video decoding
>
> Status in Chromium Browser:
> Unknown
> Status in Mozilla Firefox:
> Confirmed
> Status in chromium-browser package in Ubuntu:
> Confirmed
> Status in firefox package in Ubuntu:
> Confirmed
> Status in chromium-browser package in CentOS:
> Unknown
>
> Bug description:
> The chromium team has done a great job to totally disable hardware
> decoding on Linux.
>
> Even ignoring the GPU blacklist does not enable GPU decoding.
>
> How ever the patch is quite simple and it's already working using this
> PPA ( using libVA ) :
>
> https://launchpad.net/~saiarcot895/+archive/ubuntu/chromium-dev
>
> the corresponding patch is here :
>
> http://bazaar.launchpad.net/~saiarcot895/chromium-browser/chromium-
> browser.trusty.beta/view/head:/debian/patches/enable_vaapi_on_linux.diff
>
> that would be great if we could have this patch indefault Ubuntu
> chromium build, this situation is really sad right now, google use
> linux to make chromeOS so chromebook are really good at playing videos
> but GNU/Linux chromium is slow at doing it.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/chromium-browser/+bug/1424201/+subscriptions
>

I've recently spent some time testing the implementations in various browsers. Chromium implements VA-API but it's disabled in official builds and fragile, breaking regularly. Webkit uses gstreamer-vaapi and I've had quality issues with that using both Intel (wrong colours) and AMD (blur) hardware.

I've seen Mozilla tracks issues for VA-API and VDPAU/NVDEC. Have you seen libplacebo[1]? It encapsulates the core video rendering used in mpv in a reusable library. mpv's video acceleration has been stable and flawless for both VAAPI and NVDEC (and DXVA, FWIW, but I've no experience with that) for years. In contrast to the browsers' and gstreamer va-api implementations, I've never had performance, stability or quality issues with mpv.

In fact, hundreds (thousands?) of Firefox users use several WebExtensions to have videos played in mpv[2],[3],[4].

I haven't worked with either Firefox or mpv code so I can only speak from a user's perspective right now. I just thought, it might be worth to consider implementing libplacebo instead of rolling an own implementation for va-api and vdpau/nvdec. libplacebo is licensed in LGPLv2.1.

Note: I initially wanted to post this in #1210726, quoting first comment:

> This bug will track support for hardware decoding on Linux and the various system available (VA-API, VDPAU, DXVA and others)
>
> There's so many frameworks available, it's hard to keep track!

But this one's locked. If it suits better there, feel free to move this comment.

[1]: https://github.com/haasn/libplacebo
[2]: https://addons.mozilla.org/en-US/firefox/addon/send-to-mpv-player/
[3]: https://addons.mozilla.org/en-US/firefox/addon/ff2mpv/
[4]: https://addons.mozilla.org/en-US/firefox/addon/play-with/

Displaying first 40 and last 40 comments. View all 124 comments or add a comment.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.