Hardware-accelerated video decoding (VA-API) broken in Firefox 79 (Wayland)

[Jani Uusitalo]

After updating Firefox from 78 to 79, hardware-accelerated video decoding no longer works properly in Wayland: streaming video gets cut off with an error message from the service.

== Steps to reproduce ==
1. Follow the steps in [1] to enable VA-API.
2. Open a Youtube/Twitch video (for instance https://www.twitch.tv/rifftrax), press play

== What I expect to happen ==
For the video be played without issues (as it did with Firefox 78).

== What happens ==
After playing for a while (anything from just a few seconds to a couple of minutes), the video stops and is replaced by a service-specific error message, such as error #3000 (in case of Twitch). Even when playing, the video also flickers green. After reloading the tab, the video again plays for a few seconds before the error message reappears.

== Other info ==
A fix is apparently already in the works upstream [2].

* [1] https://mastransky.wordpress.com/2020/06/03/firefox-on-fedora-finally-gets-va-api-on-wayland/
* [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1645671

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: firefox 79.0+build1-0ubuntu0.20.04.1
ProcVersionSignature: Ubuntu 5.4.0-42.46-generic 5.4.44
Uname: Linux 5.4.0-42-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu27.4
Architecture: amd64
BuildID: 20200720193547
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Fri Jul 31 17:07:50 2020
InstallationDate: Installed on 2016-10-13 (1387 days ago)
InstallationMedia: Ubuntu-Server 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=fi_FI.UTF-8
 SHELL=/bin/bash
SourcePackage: firefox
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.apport.crashdb.conf: [modified]
mtime.conffile..etc.apport.crashdb.conf: 2020-05-11T14:52:22.308877

[W-jan-k (w-jan-k)]

Created attachment 9156561
Bildschirmfoto vom 2020-06-14 12-57-09.png

This is a minor cosmetic issue. It affects software and VAAPI.
It occurs only once. To reproduce again, reload the tab or change video resolution on YouTube.
(With media.ffvpx.enabled:false this can be observed with VP9 videos.)

software: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

hardware: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

[Marko Popović (macrox95)]

I can confirm this bug, started happening a few days ago.

[甘露 (Lu Gan) (rhythm-gan)]

I meet the similar issue with huya.com which is a live streaming website.

[Stransky (stransky)]

So this is a regression from Bug 1629788, correct?

[W-jan-k (w-jan-k)]

Correct.
https://hg.mozilla.org/integration/autoland/shortlog/0c7df6f9b0c1
first affected build: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout
last unaffected build: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 6f6757bfd3298d8314d52730c3c016dc4c629f05 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

[W-jan-k (w-jan-k)]

Same for VAAPI:
first affected build: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout
last unaffected build: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 6f6757bfd3298d8314d52730c3c016dc4c629f05 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

[Marko Popović (macrox95)]

This started happening again after today's update...

[W-jan-k (w-jan-k)]

This was never gone and the feature is not finished yet (bug 1629788 comment 27). Some other patches still await review.
VAAPI was just temporarily disabled: bug 1645706 disabled the dmabuf-video-textures pref that VAAPI now depends upon. You had to enable two prefs for VAAPI. bug 1646051 changed it to let the VAAPI pref also enable dmabuf-video-textures. You see this bug again because you are using VAAPI again.

[Marko Popović (macrox95)]

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #7)
> This was never gone and the feature is not finished yet (bug 1629788 comment 27). Some other patches still await review.
> VAAPI was just temporarily disabled: bug 1645706 disabled the dmabuf-video-textures pref that VAAPI now depends upon. You had to enable two prefs for VAAPI. bug 1646051 changed it to let the VAAPI pref also enable dmabuf-video-textures. You see this bug again because you are using VAAPI again.

Thanks for clarification! I thought it was gone for a few days.

[George Sofianos (gsf-greece)]

(In reply to Marko from comment #8)
> Thanks for clarification! I thought it was gone for a few days.

I never had this on my PC (AMD 5700XT / mesa 20.1.1 / Linux 5.7.2) and I was using VAAPI decoding for about a month. Just started happening today. Even with yesterdays nightly where I had to enable the dmabuf-video-textures pref, I was seeing no Green frames. I'm just commenting it in case it helps to identify what it is, but since it's work in progress I guess it will be resolved at some point.

[W-jan-k (w-jan-k)]

*** Bug 1650648 has been marked as a duplicate of this bug. ***

[O-mj-q (o-mj-q)]

For reference, this is also an issue on X11/EGL, even without `media.ffmpeg.dmabuf-textures.enabled`.

[Marko Popović (macrox95)]

Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?

[Mel34 (mel34)]

I agree with comment 12, now that firefox 79 is officially out this bug is very visible to anyone using hardware accelerated video playback on linux.

[Robert Mader (robert.posteo)]

(In reply to Marko from comment #12)
> Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?

First of all this feature is not yet enabled by default and requires users to jump through several hoops to enable it (enable EGL either for X11 or by using the Wayland backend, enable WR, enable several options). Secondly it's not a crash but just a glitch, and not a major one neither. Finally the feature was not developed by the FF core team but external contributors. This together results in a low priority. And no, fixes for disabled-by-default features will usually not get backported.

But don't worry, I'm sure Martin will work on it as time allows.

[Marko Popović (macrox95)]

(In reply to Robert Mader [:rmader] from comment #14)
> (In reply to Marko from comment #12)
> > Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?
>
> First of all this feature is not yet enabled by default and requires users to jump through several hoops to enable it (enable EGL either for X11 or by using the Wayland backend, enable WR, enable several options). Secondly it's not a crash but just a glitch, and not a major one neither. Finally the feature was not developed by the FF core team but external contributors. This together results in a low priority. And no, fixes for disabled-by-default features will usually not get backported.
>
> But don't worry, I'm sure Martin will work on it as time allows.

I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.

[Stransky (stransky)]

*** Bug 1645673 has been marked as a duplicate of this bug. ***

[Stransky (stransky)]

(In reply to Marko from comment #15)
> I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
> Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.

The patch is on the way. I prefer to unify X11/Wayland VA-API/dmabuf backends and then do all the fixes for both together, that lowers the maintenance cost. This is really still an experimental feature so please accept potential breakage during the development (and I also have other duties at Red Hat).

[Marko Popović (macrox95)]

(In reply to Martin Stránský [:stransky] from comment #17)
> (In reply to Marko from comment #15)
> > I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
> > Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.
>
> The patch is on the way. I prefer to unify X11/Wayland VA-API/dmabuf backends and then do all the fixes for both together, that lowers the maintenance cost. This is really still an experimental feature so please accept potential breakage during the development (and I also have other duties at Red Hat).

Thnaks for the clarification about X11/wayland common backend.

Don't worry Martin, we owe you eternal gratitude for implementing this feature in Firefox after so many years the issue was neglected by other developers!

[Stransky (stransky)]

Created attachment 9166559
Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

When DMABufSurfaceWrapper is added to nsTArray, a temporary local DMABufSurfaceWrapper object is created. When the temporary
object is deleted after the adding, associated dmabuf data is released which leads to rendering artifact during VA-API video playback.

As a fix in this patch we use UniquePtr<DMABufSurfaceWrapper> at nsTArray thus we don't create the temporary DMABufSurfaceWrapper object.
Also change GetUnusedDMABufSurfaceWrapper() to GetUnusedDMABufSurfaceWrapperIndex() and reference the surface wrapper by indexe instead of pointer.

[Stransky (stransky)]

*** Bug 1645672 has been marked as a duplicate of this bug. ***

[W-jan-k (w-jan-k)]

*** Bug 1656653 has been marked as a duplicate of this bug. ***

[W-jan-k (w-jan-k)]

*** Bug 1656777 has been marked as a duplicate of this bug. ***

[Pulsebot (pulsebot)]

Pushed by <email address hidden>:
https://hg.mozilla.org/integration/autoland/rev/bacacb384008
[Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r=jya

[Ncsoregi (ncsoregi)]

https://hg.mozilla.org/mozilla-central/rev/bacacb384008

[Release-mgmt-account-bot (release-mgmt-account-bot)]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit [auto_nag documentation](https://wiki.mozilla.org/Release_Management/autonag#missing_beta_status.py).

[W-jan-k (w-jan-k)]

[Tracking Requested - why for this release]:

This regression affects:
* MOZ_ENABLE_WAYLAND=1 users with Firefox 79,
* MOZ_X11_EGL=1 and MOZ_ENABLE_WAYLAND=1 users with Firefox 80.

Everyone is excited that Firefox 80 finally supports VAAPI hardware decoding on X11, it got news coverage, etc., but those who tested it know that it suffers from at least two fatal bugs. (bug 1645672 is the other one.) Wayland/MOZ_X11_EGL and VAAPI are still experimental and disabled-by-default, but it's a killer feature that brings users back. Therefore please consider uplifing this into beta, and even into a dot release, if there is any.

Downstream Fedora applies this patch on Firefox 79: https://src.fedoraproject.org/rpms/firefox/c/cd18e999f576523b6b3f2076bca625d5e04d1178?branch=master

https://www.reddit.com/r/archlinux/comments/i0ireu/firefox_developer_edition_hardware_video/fzpojad/
https://www.reddit.com/r/flatpak/comments/i157dm/green_flashes_in_video_in_firefox/
https://www.reddit.com/r/linux/comments/i04cvr/psa_firefox_79_appears_to_break_vaapi_decoding_on/
https://www.reddit.com/r/firefox/comments/i1p2ct/linux_vaapi_video_decode_acceleration_unreliable/

[Julien Cristau (jcristau-mozilla)]

Not tracking as this is still disabled.

[Release-mgmt-account-bot (release-mgmt-account-bot)]

The patch landed in nightly and beta is affected.
:stransky, is this bug important enough to require an uplift?
If not please set `status_beta` to `wontfix`.

For more information, please visit [auto_nag documentation](https://wiki.mozilla.org/Release_Management/autonag#uplift_beta.py).

[Stransky (stransky)]

Comment on attachment 9166559
Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

### Beta/Release Uplift Approval Request
* **User impact if declined**: Broken VA-API video playback on Wayland/X11.
* **Is this code covered by automated tests?**: No
* **Has the fix been verified in Nightly?**: Yes
* **Needs manual test from QE?**: No
* **If yes, steps to reproduce**:
* **List of other uplifts needed**: None
* **Risk to taking this patch**: Low
* **Why is the change risky/not risky? (and alternatives if risky)**: Changes a way how elements are stored in nsTArray, to in-place assignment.
* **String changes made/needed**:

[Julien Cristau (jcristau-mozilla)]

Comment on attachment 9166559
Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

wayland/vaapi fix, approved for 80.0b5

[Julien Cristau (jcristau-mozilla)]

https://hg.mozilla.org/releases/mozilla-beta/rev/8d720959430b

[Ronan Jouchet (ronj)]

Hi.

Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, `intel_gpu_top` confirms HW acceleration is active, but ***all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames)*** . Is there already a bug for it, or should I file one?

- `MOZ_X11_EGL=1`
- `media.ffmpeg.dmabuf-textures.enabled:true` , `media.ffmpeg.vaapi.enabled:true` , `media.ffvpx.enabled:false`
- enhanced-h264ify forcing H264
- Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

If already answered, sorry for the noise. If not, please tell me what debug info you'd like me to share in a new bug.

[Ronan Jouchet (ronj)]

(In reply to Ronan Jouchet from comment #32)
> Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, `intel_gpu_top` confirms HW acceleration is active, but ***all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames)*** . Is there already a bug for it, or should I file one?
>
> - `MOZ_X11_EGL=1`
> - `media.ffmpeg.dmabuf-textures.enabled:true` , `media.ffmpeg.vaapi.enabled:true` , `media.ffvpx.enabled:false`
> - enhanced-h264ify forcing H264
> - Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to `vainfo`), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.

[Ananda Laksmana (ananda-laksmana)]

(In reply to Ronan Jouchet from comment #33)
> (In reply to Ronan Jouchet from comment #32)
> > Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, `intel_gpu_top` confirms HW acceleration is active, but ***all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames)*** . Is there already a bug for it, or should I file one?
> >
> > - `MOZ_X11_EGL=1`
> > - `media.ffmpeg.dmabuf-textures.enabled:true` , `media.ffmpeg.vaapi.enabled:true` , `media.ffvpx.enabled:false`
> > - enhanced-h264ify forcing H264
> > - Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.
>
> Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to `vainfo`), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.

I can reproduce this on my system with AMDGPU driver and with same about:config entries as you.

[Ananda Laksmana (ananda-laksmana)]

(In reply to Anan Laks from comment #34)
> (In reply to Ronan Jouchet from comment #33)
> > (In reply to Ronan Jouchet from comment #32)
> > > Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, `intel_gpu_top` confirms HW acceleration is active, but ***all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames)*** . Is there already a bug for it, or should I file one?
> > >
> > > - `MOZ_X11_EGL=1`
> > > - `media.ffmpeg.dmabuf-textures.enabled:true` , `media.ffmpeg.vaapi.enabled:true` , `media.ffvpx.enabled:false`
> > > - enhanced-h264ify forcing H264
> > > - Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.
> >
> > Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to `vainfo`), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.
>
> I can reproduce this on my system with AMDGPU driver and with same about:config entries as you.

With both H264 and VP9 video.

[Jacobmee (jacobmee)]

I am experiencing the exact same issue as Ronan Jouchet and Anan Laks.

- I am on X11/intel modesetting
- I have MOZ_X11_EGL=1, media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
- I play a video, check with intel_gpu_top that HW acceleration is working (it is)
- the video is just green.

Like Ronan I have also been able to reproduce this on a completely vanilla nightly profile. This happens with both h264 and VP9 video like in Anan's case.

[Robert Mader (robert.posteo)]

It's likely that you see bug 1658035 - should be fixed by next nightly.

[Stransky (stransky)]

Please file a new bug about it and also attach a media log.
Run Firefox on terminal with MOZ_LOG="PlatformDecoderModule:5" to get the media log.
My guess is that we don't import dmabuf surfaces correctly, the decoder may use a different format or so.
Thanks.

[W-jan-k (w-jan-k)]

Yes, bug 1658035 is about videos being completely green until the next Nightly update.

[Ronan Jouchet (ronj)]

Well that was fast :D . Fix confirmed in today's Nightly 81.0a1/20200808093545, VAAPI acceleration is effective for me under Xorg/Intel, and videos are no longer completely green. Thanks for the fix, Robert, and thanks for the Xorg push, Martin!

With that, not creating a new bug. If you nevertheless still would like the info you requested in Comment 38, Martin, do ping me and I'll create one.

[Ananda Laksmana (ananda-laksmana)]

(In reply to Ronan Jouchet from comment #40)
> Well that was fast :D . Fix confirmed in today's Nightly 81.0a1/20200808093545, VAAPI acceleration is effective for me under Xorg/Intel, and videos are no longer completely green. Thanks for the fix, Robert, and thanks for the Xorg push, Martin!
>
> With that, not creating a new bug. If you nevertheless still would like the info you requested in Comment 38, Martin, do ping me and I'll create one.

Can confirm, now VAAPI acceleration is working properly on latest Nightly build.

Thanks for the fix, everyone.

[Aditya Suseno (aditya-suseno)]

This is happened to me when `widget.wayland-dmabuf-vaapi.enabled` is set to **True**