Firefox 89: webrender breaks videos on Raspberry Pi

Bug #1930982 reported by David Griffin
30
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Mozilla Firefox
Fix Released
Unknown
firefox (Ubuntu)
High
Olivier Tilloy

Bug Description

Firefox 89 enabled the Webrender framework by default. Unfortunately, the GFX drivers on a Pi 4 seem to be non-compliant in a way that breaks videos - specifically, there will be a lot of primary coloured artefacts in videos. My guess is that other Raspberry Pi's will also be affected, but I don't have the hardware available to test. Setting gfx.webrender.force-disabled to True fixes the issue, so it's fairly clear where it lies.

Mozilla doesn't seem to provide official builds for Raspberry Pi's (or Linux/Arm in general), so I think this is an issue to fix in these packages, rather than upstream. The quick solution would to just set gfx.webrender.force-disabled to True for the Pi-platforms and be done with it, but there may be better solutions I don't know about.

Additioal info:
Ubuntu release: 21.04
Firefox package: firefox/hirsute-security,hirsute-updates,now 89.0+build2-0ubuntu0.21.04.1 arm64
Steps to reproduce: Play a video on Youtube, watch the coloured blocks appear. Then set gfx.webrender.force-disabled to True and restart to verify the fix.

Revision history for this message
In , Walter ZAMBOTTI (zambotti) wrote :

Created attachment 9225141
Firefox 89.0 black video artifacts.png

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0

Steps to reproduce:

Updated to 89.0

Actual results:

Playing any video video content which contains black causes red and blue artifacts in where the lack should be.

Expected results:

Expect to have seen black instead of red and blue. A picture/screenshot is worth a thousand words and is attached.

Revision history for this message
In , Walter ZAMBOTTI (zambotti) wrote :

Name Firefox
Version 89.0
Build ID 20210527174632
Distribution ID canonical
User Agent Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0
OS Linux 4.9.241-77 #1 SMP PREEMPT Mon Mar 29 23:04:14 -03 2021

Revision history for this message
In , Release-mgmt-account-bot (release-mgmt-account-bot) wrote :

The [Bugbug](https://github.com/mozilla/bugbug/) bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

David Griffin (habilain)
description: updated
Revision history for this message
Mike Feign (mikefeign) wrote :

Yes I have the same issue when I install Ubuntu desktoop 21.04.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in firefox (Ubuntu):
status: New → Confirmed
Revision history for this message
Maik (maik-adamietz) wrote :

I also experience this issue on the Raspberry Pi 4 with Firefox 89 on Ubuntu 20.10.

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Created attachment 9225490
firefox_video.png

I'm also having this issue, also on aarch64, a Pinebook Pro:

I'm seeing RPi users with a similar issue.

Name Firefox
Version 89.0
Build ID 20210602080605
Distribution ID manjaro
User Agent Mozilla/5.0 (X11; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0
OS Linux 5.12.9-1-MANJARO-ARM #1 SMP Thu Jun 3 08:06:55 UTC 2021

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Created attachment 9225491
about_support_raw.json

Here's my about:support as raw json.

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Disabling WebRender as suggested here gets rid of the artifacting, but presumably this is still a problem.

https://support.mozilla.org/en-US/questions/1338832

"about:config => gfx.webrender.force-disabled = true"

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Err to be perfectly clear, disabling WebRender, and then restarting Firefox fixed the issue, in that order.

Revision history for this message
In , Alwu (alwu) wrote :

Per comment6, this seems related with Graphic.

Revision history for this message
In , Jnicol-x (jnicol-x) wrote :

Yep, seems like an issue with software webrender playing videos on ARM Linux.

See also bug 1714739 (probably a dup of this)

Sotaro, any ideas?

Revision history for this message
In , Jnicol-x (jnicol-x) wrote :

*** Bug 1714739 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Jnicol-x (jnicol-x) wrote :

Walter, Conrad, does this occur on other video sites too, or just youtube? And is it every video on youtube (Perhaps it only affects certain video formats)?

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Jamie, I can confirm it happens on other sites, it also happens regardless of the codec as far as I can tell - normally I have everything but avc1/h264 disabled on the device in about:config just due to it being kinder on battery life, but I re-enabled vp9 (and ensured that youtube, in this case, was serving vp9) and it has the same issue. Please let me know if there's anything else you'd like me to check.

Revision history for this message
In , Walter ZAMBOTTI (zambotti) wrote :

Jamie Nicol asked :

Walter, Conrad, does this occur on other video sites too, or just youtube? And is it every video on youtube (Perhaps it only affects certain video formats)?

It appears to be happening on all sites that contain videos even video adds!

Revision history for this message
In , Jmuizelaar (jmuizelaar) wrote :

Lee, it looks like there might be an issue with our YUV conversion code on ARM. Any guesses what that might be?

Revision history for this message
In , Jmuizelaar (jmuizelaar) wrote :

Another thing to try would be enabling hw-wr. You should be able to do that by setting gfx.webrender.all=true and unsetting gfx.webrender.force-disabled. I expect this will improve performance, battery life and remove the artifacts.

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Created attachment 9226348
webrender_all_true_force_disable_false.png

Unfortunately no luck there, that seems to have caused Firefox to suddenly come to life with creativity/inspiration for the visual arts.

I am running the "bleeding edge stable" of the Panfrost driver with OpenGL 3.3 enabled (I believe the latter part specifically is experimental); no idea if that has any bearing on this but I'll disable that later and see if that makes any difference. Appreciate the suggestions (and possible hints at improved power consumpiton in the future on aarch64/in general by default).

Cheers, and again if there's anything else I can test I'd be happy to help.

Revision history for this message
In , Conrad-w (conrad-w) wrote :

These new artifacts with gfx.webrender.all=true and gfx.webrender.force-disabled=false occurred with both my compositor running and not (still using X11).

Revision history for this message
In , Jmuizelaar (jmuizelaar) wrote :

Ok yeah, those new artifacts could be panfrost bugs. They're probably worth reporting to upstream mesa.

Revision history for this message
In , Lsalzman (lsalzman) wrote :

Can you attach your "about:buildconfig" so we can see what compiler setup you built Firefox with?

Revision history for this message
In , Walter ZAMBOTTI (zambotti) wrote :

(In reply to Jeff Muizelaar [:jrmuizel] from comment #17)
> Ok yeah, those new artifacts could be panfrost bugs. They're probably worth reporting to upstream mesa.

They could be panfrost issues but then we need to explain why the the previous version of Firefox is working fine with the same panfrost drivers.

In addition I have two ARM Linux systems. One is 18.04 LTS X11 the other is 20.04 PANFROST.

So whether there is panfrost or not the problem still exists.

I think we can safely discount PANFROST at this time.

Revision history for this message
In , Sotaro-ikeda-g (sotaro-ikeda-g) wrote :

(In reply to Jamie Nicol [:jnicol] from comment #8)
> Yep, seems like an issue with software webrender playing videos on ARM Linux.
>
> See also bug 1714739 (probably a dup of this)
>
> Sotaro, any ideas?

I could reproduce the problem with sw-wr on ARM64 Win10. Then it seems arm specific problem.

Revision history for this message
In , Conrad-w (conrad-w) wrote :

Created attachment 9226630
about_buildconfig.html

Re: mesa, I can confirm that my mesa version did not change between firefox versions. I tried playing around with downgrading things but didn't have any luck.

Here's my about:buildconfig, wasn't sure how to format so just saved the html.

Revision history for this message
In , Lsalzman (lsalzman) wrote :

Created attachment 9226716
Bug 1714511 - Use more saturated ops to avoid YUV math underflow. r?sotaro

Revision history for this message
In , Lsalzman (lsalzman) wrote :

Sotaro, if you are able to repro, can you check if my saturated math patch helps the issue for you?

Revision history for this message
In , Sotaro-ikeda-g (sotaro-ikeda-g) wrote :

(In reply to Lee Salzman [:lsalzman] from comment #23)
> Sotaro, if you are able to repro, can you check if my saturated math patch helps the issue for you?

I tried the following build. The problem was not addressed with D117599.
https://treeherder.mozilla.org/jobs?repo=try&revision=99c8dcb3ae16b3d3f19bcef6dfbc7ac783853e0f

Dave Jones (waveform)
tags: added: raspi-image
Changed in firefox (Ubuntu):
importance: Undecided → High
tags: added: rls-ii-incoming
tags: added: impish
Revision history for this message
Olivier Tilloy (osomon) wrote :

I can confirm the issue on a a rpi4 running 21.04 (arm64). The suggested workaround (setting gfx.webrender.force-disabled to True) works.

Changed in firefox (Ubuntu):
assignee: nobody → Olivier Tilloy (osomon)
Changed in firefox:
status: Unknown → New
Revision history for this message
In , Lsalzman (lsalzman) wrote :

The circumstantial evidence "seems like" something in the function here (https://searchfox.org/mozilla-central/source/gfx/wr/swgl/src/composite.h#534) is underflowing. But I would need someone who can reliably reproduce the issue to find out where in those math equations are actually doing so. I tried testing in my patch if the additions were the culprit, but that seems unlikely.

It's possible the multiplications "rbCoeffs * uv" or "gCoeffs * uv" are underflowing based on whatever values the ARM video codecs seem to be supplying. However, this is not happening on x86 at all, which is strange, and it is also happening on ARM across platform, which is further strange, since there isn't really much in the way of platform specific code here.

Sotaro, might you be able to investigate this?

Revision history for this message
In , Lsalzman (lsalzman) wrote :

Created attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

Revision history for this message
In , Lsalzman (lsalzman) wrote :

Sotaro, can you see if my vqmovun_s16 patch fixes this or not?

Revision history for this message
In , Walter ZAMBOTTI (zambotti) wrote :

(In reply to Lee Salzman [:lsalzman] from comment #25)
> The circumstantial evidence "seems like" something in the function here (https://searchfox.org/mozilla-central/source/gfx/wr/swgl/src/composite.h#534) is underflowing. But I would need someone who can reliably reproduce the issue to find out where in those math equations are actually doing so. I tried testing in my patch if the additions were the culprit, but that seems unlikely.
>
> It's possible the multiplications "rbCoeffs * uv" or "gCoeffs * uv" are underflowing based on whatever values the ARM video codecs seem to be supplying. However, this is not happening on x86 at all, which is strange, and it is also happening on ARM across platform, which is further strange, since there isn't really much in the way of platform specific code here.
>
> Sotaro, might you be able to investigate this?

Taking a step back. If a certain function is suspected what are the diff's with this code between the working 88.0 and non working 89.0 version?

Since the issue affects all ARM regardless of platform the only culprit not eliminated is Firefox.

If you have a downloaded executable for me to test I'm happy to do so. My platforms are :

aarch64 Ubuntu 18.04 LTS
aarch64 Ubuntu 20.10

I can only test on one of these platforms!

Revision history for this message
In , Sotaro-ikeda-g (sotaro-ikeda-g) wrote :

(In reply to Lee Salzman [:lsalzman] from comment #27)
> Sotaro, can you see if my vqmovun_s16 patch fixes this or not?

The updated patch worked for me on arm64 Win10 :)

Revision history for this message
In , Pulsebot (pulsebot) wrote :

Pushed by <email address hidden>:
https://hg.mozilla.org/integration/autoland/rev/c77fc5fac5ae
use vqmovun_s16 for packing pixels. r=sotaro

Revision history for this message
In , Mlaza (mlaza) wrote :
Revision history for this message
In , Olivier Tilloy (osomon) wrote :

Tested that commit on top of the 89.0 release, on Ubuntu 21.04 on aarch64 (Raspberry Pi 4), and I can confirm the problem is fixed. Thanks a lot :lsalzman ! I'm going to cherry-pick that commit as a distro-patch for Ubuntu packages.

Revision history for this message
Olivier Tilloy (osomon) wrote :

Fix committed upstream: https://hg.mozilla.org/mozilla-central/rev/c77fc5fac5ae.
I'm going to cherry-pick it as a distro-patch.

Olivier Tilloy (osomon)
Changed in firefox (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
In , Lsalzman (lsalzman) wrote :

Comment on attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

### Beta/Release Uplift Approval Request
* **User impact if declined**: Enabling Software-WebRender on ARM platforms can lead to visual artifacts. Should not impact x86 platforms or non-SW-WR configurations.
* **Is this code covered by automated tests?**: No
* **Has the fix been verified in Nightly?**: Yes
* **Needs manual test from QE?**: No
* **If yes, steps to reproduce**:
* **List of other uplifts needed**: None
* **Risk to taking this patch**: Low
* **Why is the change risky/not risky? (and alternatives if risky)**: Software-WebRender on ARM is not deployed by default to release. Must be pref'd on. However, many users on tier-2 platforms like ARM/Linux are pref'ing the feature on, and it would be nice if we just supplied the fix to them so they don't need to manually apply it to their build.
* **String changes made/needed**:

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package firefox - 89.0.1+build1-0ubuntu1

---------------
firefox (89.0.1+build1-0ubuntu1) impish; urgency=medium

  * New upstream release (89.0.1+build1)

  * Cherry-pick an upstream commit to fix video rendering on arm (LP: #1930982)
    - debian/patches/upstream-c77fc5fac5ae.patch

 -- Olivier Tilloy <email address hidden> Tue, 15 Jun 2021 17:10:28 +0200

Changed in firefox (Ubuntu):
status: In Progress → Fix Released
Changed in firefox:
status: New → Fix Released
Revision history for this message
In , Julien Cristau (jcristau-mozilla) wrote :

Comment on attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

approved for 90.0b9

Revision history for this message
In , Julien Cristau (jcristau-mozilla) wrote :
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.