Incomplete screen redraws in virtual machines running Xorg

[Daniel van Vugt]

[ Impact ]

When using software rendering on Xorg (so usually just in VMs), some parts of app windows may fail to redraw.

https://gitlab.gnome.org/GNOME/mutter/-/issues/2880

[ Test Plan ]

1. Set up a virtual machine without graphics acceleration.
2. Log into 'Ubuntu on Xorg'.
3. Open Settings > About and verify Graphics = llvmpipe (or "Software Rendering"), and Windowing System = X11.
4. sudo snap install chromium
5. Open Chromium and find a web page with lots of text. Plain text without any links and staying at the top of the page will work best because you don't want to trigger any scrolling.
6. Start selecting text with the mouse and while holding the button move the mouse up and down rapidly. Avoid triggering any page scrolling.
7. Release the mouse button in roughly the middle of the area of text that was selected.
8. Verify the remaining text selection is contiguous and not broken into multiple disconnected text selections (the bug).
9. Repeat steps 7 and 8 several times to be sure.

[ Where problems could occur ]

The fix adds a new synchronization point between the compositor and X server. This should be inconsequential, although there is a "sync ring" in mutter used for this, and if a mistake occurs there then other parts of the desktop could appear to stop responding in Xorg sessions.

[ Other Info ]

This is a regression that started in 3.37.3 due to https://gitlab.gnome.org/GNOME/mutter/-/commit/551101c65cda.

[Daniel van Vugt (vanvugt)]

Fixed upstream in mutter 46:
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/3590

[Daniel van Vugt (vanvugt)]

Noble will get the fix automatically with mutter 46.0

Jammy is the next most important release (especially to the customer) and is under review here:
https://salsa.debian.org/gnome-team/mutter/-/merge_requests/120

[Daniel van Vugt (vanvugt)]

Here's the jammy debdiff for both bug 2054510 and bug 2051074.

[Daniel van Vugt (vanvugt)]

Here's the mantic debdiff for both bug 2054510 and bug 2051074.

[Łukasz Zemczak (sil2100)]

Hello Daniel, or anyone else affected,

Accepted mutter into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mutter/45.2-0ubuntu4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Daniel, or anyone else affected,

Accepted mutter into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mutter/42.9-0ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Daniel van Vugt (vanvugt)]

Stange. Jammy is failing to build for a reason unrelated to anything that has changed since the last update:

In file included from ../src/backends/meta-backend-private.h:39,
                 from ../src/backends/x11/meta-backend-x11.h:31,
                 from ../src/backends/x11/nested/meta-backend-x11-nested.h:25,
                 from ../src/tests/meta-backend-test.h:23,
                 from ../src/tests/meta-backend-test.c:22:
../src/backends/meta-monitor-manager-private.h:41:10: fatal error: meta/meta-enum-types.h: No such file or directory
   41 | #include "meta/meta-enum-types.h"
      | ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

EDIT: Moved to bug 2055530

[Daniel van Vugt (vanvugt)]

Mantic built OK, except on arm64 and armhf due to known flaky tests. We can try rebuilding for those.

Jammy failed to build mostly due to a new bug 2055519 (unrelated to our changes). So we should add a fix for that and roll a new version. amd64 also failed with "fatal error: meta/meta-enum-types.h: No such file or directory", which I cannot reproduce locally. I can only reproduce bug 2055519 on jammy.

[Daniel van Vugt (vanvugt)]

Verified fixed on Mantic using Mutter version 45.2-0ubuntu4.

[Daniel van Vugt (vanvugt)]

Oh I see the issue with 'meta-enum-types.h'. It's generated at build time and most likely someone forgot to declare dependencies on that. So at random times, depending on how parallel your build is, it might not be built before it gets included. --> bug 2055530

[Daniel van Vugt (vanvugt)]

Here's the fixes for jammy build failures in 42.9-0ubuntu6

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted mutter (45.2-0ubuntu4) for mantic have finished running.
The following regressions have been reported in tests triggered by the package:

mutter/45.2-0ubuntu4 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/mantic/update_excuses.html#mutter

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Daniel van Vugt (vanvugt)]

That failing test case:

  1180s # FAIL: mutter-13/minimized.test (Child process exited with code 133)

is known to be flaky. I've seen it fail on unrelated changes upstream. And it is indeed unrelated to these changes.

The rebuild has now succeeded:

  https://launchpad.net/ubuntu/+source/mutter/45.2-0ubuntu4

[Robie Basak (racb)]

Hello Daniel, or anyone else affected,

Accepted mutter into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mutter/42.9-0ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Daniel van Vugt (vanvugt)]

Verified fixed on jammy using mutter version 42.9-0ubuntu7 and the test plan in the bug description.

[Andreas Hasenack (ahasenack)]

There is an armhf DEP8 failure that I just retriggered, let's see if it passes in the next run.

[Andreas Hasenack (ahasenack)]

The test failure on armhf persists:

939s # Running test: mutter-13/fullscreen-maximize.test
940s # Executing: mutter-13/fullscreen-maximize.test
941s # FAIL: mutter-13/fullscreen-maximize.test (Child process exited with code 1)
941s not ok - mutter-13/fullscreen-maximize.test

[Daniel van Vugt (vanvugt)]

That test passed in the actual build:
https://launchpadlibrarian.net/719148617/buildlog_ubuntu-jammy-armhf.mutter_42.9-0ubuntu7_BUILDING.txt.gz

It is known to be a flaky test (moreso on Arm) and its failure is not related to the code changes here.
https://gitlab.gnome.org/GNOME/mutter/-/issues/3050

Other examples of the same test failing upstream:
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1441#note_1984247

Also Debian already marked the test as flaky, and Ubuntu does the same, but only in GNOME 45 onward:
https://salsa.debian.org/gnome-team/mutter/-/commit/98302806f85ffe4cd67768fe63992ba32e77146a

[Kai-Heng Feng (kaihengfeng)]

Should [0] be part of the SRU to avoid the test failure?

[0] https://salsa.debian.org/gnome-team/mutter/-/commit/98302806f85ffe4cd67768fe63992ba32e77146a

[Daniel van Vugt (vanvugt)]

It can be put into the next mutter update but there is no reason to delay the currently proposed update for it. This seems like a never-ending problem that we're discovering old intermittent build failures in an old GNOME release. We will fix them as they are discovered, but not all of them need to be treated as blockers.

[Daniel van Vugt (vanvugt)]

Done: https://salsa.debian.org/gnome-team/mutter/-/commit/1e2c8fedb0bcd293501420390c3bea48f9049301

[Kai-Heng Feng (kaihengfeng)]

Thanks for the fix!

[Daniel van Vugt (vanvugt)]

Sorry I only just figured out that comment #17 is about mantic, not jammy.

https://ubuntu-archive-team.ubuntu.com/proposed-migration/mantic/update_excuses.html#mutter

And the Debian fix I mentioned in comment #18 is actually in 44.8-1 onward *and* 45.2-2 onward. So mantic would get the fix for fullscreen-maximize.test automatically if it was to receive an update to 45.3 or even if it was rebased on 45.2-2.

[Chris Halse Rogers (raof)]

Yes, please make sure the flaky tests are marked as such in the autopkgtests. SRUs are not released until either the test passes or the package version is marked as badtest in the britney hints.

(A retry has caused it to pass on armhf)

[Launchpad Janitor (janitor)]

This bug was fixed in the package mutter - 45.2-0ubuntu4

---------------
mutter (45.2-0ubuntu4) mantic; urgency=medium

  [ Kai-Heng Feng ]
  * Fix mirror mode on reduced blanking panel. (LP: #2051074)

  [ Daniel van Vugt ]
  * Add compositor-x11-Sync-again-at-the-end-of-before_paint.patch
    to fix incomplete redraws in X11 virtual machines (LP: #2054510)

 -- Daniel van Vugt <email address hidden> Thu, 22 Feb 2024 16:32:33 +0800

[Timo Aaltonen (tjaalton)]

The verification of the Stable Release Update for mutter has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mutter - 42.9-0ubuntu7

---------------
mutter (42.9-0ubuntu7) jammy; urgency=medium

  * Add build-Set-built-headers-as-libmutter_dep-sources.patch to prevent
    occasional build failures depending on parallelism (LP: #2055530)
  * Add cogl-gl-framebuffer-Fix-inverted-test-in-ensure_bits_init.patch to
    fix test failures in mutter:cogl+cogl/conform / framebuffer-get-bits
    when Mesa >= 23.1.1 (LP: #2055519)

mutter (42.9-0ubuntu6) jammy; urgency=medium

  [ Kai-Heng Feng ]
  * Fix mirror mode on reduced blanking panel. (LP: #2051074)

  [ Daniel van Vugt ]
  * Add compositor-x11-Sync-again-at-the-end-of-before_paint.patch
    to fix incomplete redraws in X11 virtual machines (LP: #2054510)
  * debian/control.in: Stop automatically reverting to the old uploaders.

 -- Daniel van Vugt <email address hidden> Fri, 01 Mar 2024 17:47:40 +0800

[Egmont Koblinger (egmont-gmail)]

During the last barely more than 24 hours, at least 4 people have reported/confirmed heavy latencies newly appearing in their GNOME Terminal. One person with Ubuntu 23.10 and two people with Ubuntu 22.04 (one of which comments has just been removed).

https://gitlab.gnome.org/GNOME/gnome-terminal/-/issues/8070

https://askubuntu.com/questions/1509058/input-delay-on-terminal-ubuntu-22-04-4

GNOME Terminal, and its underlying VTE terminal emulator widget, did not receive any updates recently in Ubuntu, so it's highly suspected that the bug sneaked in into some other component (or maybe a valid change in some other component triggered a yet undiscovered bug in VTE / GNOME Terminal).

Even though I cannot reproduce the issue myself, I went through the log of my recent apt updates, and this mutter change is the only one seemingly relevant (see https://gitlab.gnome.org/GNOME/gnome-terminal/-/issues/8070#note_2067801 onwards); in fact, the summary showing a suspiciously high correlation with the new problem.

At this point I don't have anything concrete, and it might just be a random coincidence, but I highly suspect that this recent mutter update introduces severe lagging in GNOME Terminal (and other terminal emulators using VTE's GTK3 version).

[Egmont Koblinger (egmont-gmail)]

Another most likely duplicate: https://github.com/gnome-terminator/terminator/issues/899

And yet another most likely duplicate: https://bbs.archlinux.org/viewtopic.php?id=294164. Here a user claims that downgrading gnome-shell and mutter fixes the issue, and points to upstream issue https://gitlab.gnome.org/GNOME/mutter/-/issues/3384.

Based on that report, some non-VTE-based apps are also affected.

It really looks like the bug is in upstream Mutter, and made it into downstream updates as well.

[Daniel van Vugt (vanvugt)]

The above issue is being tracked in bug 2059847.

[Marsh (marsh)]

Update on Jammy yesterday installed libmutter version 42.9-0ubuntu7.1 and the slow behavior has returned. Shoulda pinned the ppa version.

[Daniel van Vugt (vanvugt)]

You're talking about bug 2059847, not this one.