Missing VA-API hardware decoding support drains battery

[Paul Menzel]

Current devices ship with graphics devices supporting hardware decoding of certain video formats. While doing that, energy can be saved, as the main processor can go to sleep, and the graphics devices do their jobs more efficiently saving energy.

Today, the browser is used a lot to watch movies. Unfortunately, the Firefox browser does not support VA-API hardware decoding, which is the interface to be used for Intel graphics device, which are included in current Intel processors.

The issue in the Mozilla bug tracker [1] is open for over two years.

Currently, this missing feature is a major show stopper for people using Ubuntu GNU/Linux on their laptops, as compared to Microsoft Windows one battery charge(?) lasts noticeably less time.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1210727

[Jyavenard-9 (jyavenard-9)]

This would do decoding only ; not rendering (yet)

[RussianNeuroMancer (russianneuromancer)]

Hardware decoding on Intel Ironlake should be used only for 720p and lower.
Hardware 1080p decoding on Intel Ironlake is slower than software decoding on CPU: https://bugs.freedesktop.org/show_bug.cgi?id=47858

GPU model from about:support
Intel Open Source Technology Center -- Mesa DRI Intel(R) Ironlake Mobile

I need to fill separate issue about VA-API on Ironlake or drop this info here is fine?

[Jyavenard-9 (jyavenard-9)]

(In reply to russianneuromancer from comment #1)
> Hardware decoding on Intel Ironlake should be used only for 720p and lower.
> Hardware 1080p decoding on Intel Ironlake is slower than software decoding
> on CPU: https://bugs.freedesktop.org/show_bug.cgi?id=47858
>
> GPU model from about:support
> Intel Open Source Technology Center -- Mesa DRI Intel(R) Ironlake Mobile
>
> I need to fill separate issue about VA-API on Ironlake or drop this info
> here is fine?

We have a generic preference to disable hardware acceleration, I'll hook it up to vaapi, but I have no intention at this stage to make specific cases for particular chipset.

Would have to be in a separate bug.

[Ajones-m (ajones-m)]

Mass change P2 -> P3

[[e-mail] (sheepdestroyer-gmail)]

Is there a reason the priority would go from P2 to P3?

Lack of hardware acceleration is the worse offender on Linux right now. Some youtube videos can not even play at a decent framerate because of that, especially the 4K / 360 ones. And that's without speaking of the heat generated by those poor CPUs...

I do not get why it would not even go the other way, right to P1 ?

[Ajones-m (ajones-m)]

See bug 1210726.

[agm97 (albertogomezmarin)]

linux had have the firefox browser by default in all distros I think much users of firefox are from linux and the only thing we get from mozilla a long waiting for a feature that is very needed.. between google with chromium and the denied featured requests and firefox not to working on features for firefox, linux in browser category is very late and the battery for laptops is damaged with linux and video viewing with browsers, I feel really annoying

[Melroy van den Berg (webmaster-web-share)]

I fully agree, video acceleration is a must have feature now a days. Please solve it! Don't force me to move to Chrome....

[Waz (paviluf)]

I still have hope too to see full VDPAU/VAAVI support in order to have hardware video acceleration.

[Ehumphries (ehumphries)]

Gentle reminder:

* Our guidelines ask you don't argue about priority setting on bugs. Triage leads and Mozillians working on the bug set the priority. See: https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
* "Me too" or "Fix this or I'm moving to Chrome/Lynx/Gopher/Edge/Webkit" comments are off topic and just add noise.

[Laurent Bonnaud (laurent-bonnaud)]

Hi,

in firefox release/51 and beta/52 in about:support one can see "Hardware H264 Decoding" status.

However in firefox aurora/53 and nigthly/54, this line is no longer visible.

Was this done on purpose or is this an omission?

BTW, how about adding a line about "Hardware VP9 Decoding" since recent Intel GPU do support hardware VP9 decoding?

[Pmenzel+bugzilla-mozilla-org (pmenzel+bugzilla-mozilla-org)]

(In reply to Laurent Bonnaud from comment #10)
> in firefox release/51 and beta/52 in about:support one can see "Hardware
> H264 Decoding" status.
>
> However in firefox aurora/53 and nigthly/54, this line is no longer visible.
>
> Was this done on purpose or is this an omission?
>
> BTW, how about adding a line about "Hardware VP9 Decoding" since recent
> Intel GPU do support hardware VP9 decoding?

Laurent, if this is still reproducible – I only have 45.9.0 here –, could you please create a separate report for this issue? That’d be great.

[Pmenzel+bugzilla-mozilla-org (pmenzel+bugzilla-mozilla-org)]

@ehumphries, if I interpret the bug comments correctly, you flagged this report with *NEEDINFO*. Could you please tell us, what information is needed to get the feature implemented?

[Mhoye-4 (mhoye-4)]

"needinfo" is intended to call the attention of a specific person to the bug - I need information from X, to answer question Y.

It's not intended as a nonspecific "we need more information to figure this out" thing, but it's true the term is a bit opaque.

[Lordxraven (lordxraven)]

(In reply to Paul Menzel from comment #11)
> Laurent, if this is still reproducible – I only have 45.9.0 here –, could
> you please create a separate report for this issue? That’d be great.

Is there already a separate report for this issue? There are also some other useful informations gone (i can't remember exactly, but e.g. regarding OpenGL acceleration).

[Jyavenard-9 (jyavenard-9)]

Yes, we also need accelerated layers to be able to render the frames. And right now, this has taken a set back and the feature has been disabled for now.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[89c51 (barz621)]

(In reply to Jean-Yves Avenard [:jya] from comment #15)
> Yes, we also need accelerated layers to be able to render the frames. And
> right now, this has taken a set back and the feature has been disabled for
> now.

Have you started doing work on this and you wait for accelerated layers to land in order to merge it or you will start after the layers?

[ojab (ojab)]

Just FYI: https://chromium-review.googlesource.com/c/532294 this will be [chrome-parity] soon

[W-jan-k (w-jan-k)]

"The Quantum Flow (QF) initiative is a cross-component effort to improve user perceived responsiveness and performance."
Could you please triage this if it would meet your demands for 57 or 58?

Video hardware decoding is very desirable for Linux devices that:
* do not have a powerful CPU
* should be energy-efficient (laptops)
* want to fluently play 4K videos on YouTube while using the CPU for compiling Nightly. ;-)

There is new code from Chromium in comment 17 providing VAAPI hardware video decoding that could and should be shared, or not?
If Mozilla and Google both would fix bugs it would result in far better stability for all users. Would it be possible?
At least we have example code now.

[W-jan-k (w-jan-k)]

(In reply to Jean-Yves Avenard [:jya] from comment #15)
> Yes, we also need accelerated layers to be able to render the frames.

Could we say we have a dependency to WebRender (instead of the current painting code) here, to prevent double work?

[Erdem U. Altinyurt (spamjunkeater)]

Personally, I do NOT care about accelerated layers for rendering frames. Who cares?
I just want to use "accelerated video decoding via HW" because solving the video codec exhaust many CPUs, not rendering frames after. Yes rendering via OGL will probably give much better results but who cares? We need accelerated youtube videos ASAP.

Also, read the first reply of :jya at the topic,
"for just decoding, not rendering (yet)."

I believe, after 2 years, devs need to complete first step using current painting stack. Because this gonna hurt firefox project much more than other bugs. Since linux users cannot watch simple videos with FF, they gonna switch to Chrome soon.

[sam tygier (samtygier)]

There is a bounty for this issue at https://www.bountysource.com/issues/55506502-add-va-api-hardware-decoding-support-on-linux

[Jonathan Watt (jwatt)]

Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)

[Robert Mader (robert.posteo)]

The dependency of this bug could probably changed from ogl-linux-beta to bug 1491303, as webrender will replace all the old rendering architecture and is already enabled on linux + intel in nightly.

[Robert Mader (robert.posteo)]

Although this bug is only about decoding support, I'd like to leave a note about the rendering: AFAIK rendering without any slow copies requires DMABUF support, so processes can share resources on the GPU.
For the Wayland backend this about to land, see bug 1552590, quote:

> Wayland dmabuf surface are located in GPU and can be attached as EGLImage or wl_buffer. It allows direct rendering to GPU and share the HW buffer across processes.

Also see bug 1010527

[Grmat (grmat)]

I've recently spent some time testing the implementations in various browsers. Chromium implements VA-API but it's disabled in official builds and fragile, breaking regularly. Webkit uses gstreamer-vaapi and I've had quality issues with that using both Intel (wrong colours) and AMD (blur) hardware.

I've seen Mozilla tracks issues for VA-API and VDPAU/NVDEC. Have you seen libplacebo[1]? It encapsulates the core video rendering used in mpv in a reusable library. mpv's video acceleration has been stable and flawless for both VAAPI and NVDEC (and DXVA, FWIW, but I've no experience with that) for years. In contrast to the browsers' and gstreamer va-api implementations, I've never had performance, stability or quality issues with mpv.

In fact, hundreds (thousands?) of Firefox users use several WebExtensions to have videos played in mpv[2],[3],[4].

I haven't worked with either Firefox or mpv code so I can only speak from a user's perspective right now. I just thought, it might be worth to consider implementing libplacebo instead of rolling an own implementation for va-api and vdpau/nvdec. libplacebo is licensed in LGPLv2.1.

Note: I initially wanted to post this in #1210726, quoting first comment:

> This bug will track support for hardware decoding on Linux and the various system available (VA-API, VDPAU, DXVA and others)
>
> There's so many frameworks available, it's hard to keep track!

But this one's locked. If it suits better there, feel free to move this comment.

[1]: https://github.com/haasn/libplacebo
[2]: https://addons.mozilla.org/en-US/firefox/addon/send-to-mpv-player/
[3]: https://addons.mozilla.org/en-US/firefox/addon/ff2mpv/
[4]: https://addons.mozilla.org/en-US/firefox/addon/play-with/

[Moz-m (moz-m)]

Also, please note that this would save the users a few GWh of electric energy and thus help protect the climate.

[Akarshanbiswas (akarshanbiswas)]

@grmat libplacebo is a renderer not a hardware decoder and is unfit to be used in a browser. First thing we need dmabuf support so that we can import decoded frames without copying them back to the system RAM and then display it on the screen. If I understand it correctly this is the toughest part.

[Robert Mader (robert.posteo)]

That should be bug 1572697 for Wayland.

[Stransky (stransky)]

HW decoding support has landed at Bug 1616185.

*** This bug has been marked as a duplicate of bug 1616185 ***

[lilydjwg (lilydjwg)]

I'm glad to here that, but what about support for Firefox on X11? Is it given up?

[Stransky (stransky)]

(In reply to lilydjwg from comment #30)
> I'm glad to here that, but what about support for Firefox on X11? Is it given up?

Hardware decoding is platform independent so it generally works under X11 as it's implemented as Bug 1616185.
For whole playback chain see Bug 1610199 what's needed to be done fox X11.
I was closing this one as it refers to HW decode only, not the playback.
The overall video playback is tracked at Bug 1210726.

[Stransky (stransky)]

(In reply to lilydjwg from comment #30)
> I'm glad to here that, but what about support for Firefox on X11? Is it given up?

Filed Bug 1619523 for it.

[Jyavenard-9 (jyavenard-9)]

re-opening as a meta bug for VA-API implementation (including X11)

[Eric Riese (jebuswankel)]

Reminder that there is still a bounty for this issue at BountySource

https://www.bountysource.com/issues/55506502-add-va-api-hardware-decoding-support-on-linux

I got an email from BountySource that they're going to start taking old bounties for themselves starting *2020-07-1* (reproduced below)

In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.

```
Hi,
You are receiving this email because we are updating the Bountysource Terms of Service, effective 1st July 2020.

What's changing?
We have added a Time-Out clause to the Bounties section of the agreement:

2.13 Bounty Time-Out.
If no Solution is accepted within two years after a Bounty is posted, then the Bounty will be withdrawn and the amount posted for the Bounty will be retained by Bountysource. For Bounties posted before June 30, 2018, the Backer may redeploy their Bounty to a new Issue by contacting <email address hidden> before July 1, 2020. If the Backer does not redeploy their Bounty by the deadline, the Bounty will be withdrawn and the amount posted for the Bounty will be retained by Bountysource.

You can read the full Terms of Service here

What do I need to do?
If you agree to the new terms, you don't have to do anything.

If you have a bounty posted prior to June 30, 2018 that is not currently being solved, email us at <email address hidden> to redeploy your bounty. Or, if you do not agree with the new terms, please discontinue using Bountysource.

Thanks for reading

Bountysource Team
```

[Robert Mader (robert.posteo)]

(In reply to eric.riese from comment #34)
>
> In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.

That would certainly be Martin Stránský :)
Martin, do you want to claim the bounty? If you don't care about it / don't need it yourself, you could donate it or so.

[Stransky (stransky)]

(In reply to Robert Mader [:rmader] from comment #35)
> (In reply to eric.riese from comment #34)
> >
> > In my opinion the bounty was already earned when this issue was resolved but I don't know who gets to be the arbiter.
>
> That would certainly be Martin Stránský :)
> Martin, do you want to claim the bounty? If you don't care about it / don't need it yourself, you could donate it or so.

Okay, I'll try to claim it and send it to a charity.

[Stransky (stransky)]

This needs to be closed to claim the bounty at https://www.bountysource.com/issues/55506502-meta-add-va-api-hardware-decoding-support-on-linux so closing this one.

Wayland va-api decoding was implemented by Bug 1616185
X11 variant is pending at Bug 1619523

[Mathew Hodson (mhodson)]

This should depend on bug 1616185 and bug 1619523 so it's easier to find where the work happened.