Backport packages for 22.04.4 HWE stack

[Timo Aaltonen]

[Impact]
The graphics HWE stack from mantic needs to be backported for 22.04.4

directx-headers
- build-dep of the new Mesa

mesa
- new major release (23.2.x)
- new HW support, Meteor Lake..

[Test case]
We want to cover at least 2-3 different, widely used and already previously supported GPU generations from both AMD and Intel which are supported by this release, as those are the ones that cover most bases; nouveau users tend to switch to the NVIDIA blob after installation. No need to test ancient GPU's supported by mesa-amber. And best to focus on the newer generations (~5y and newer) as the older ones are less likely to break at this point.
- AMD: Vega, Navi1x (RX5000*), Navi2x (RX6000*), Navi3x (RX7000*)
- Intel: gen9 (SKL/APL/KBL/CFL/WHL/CML), gen11 (ICL), gen12 (TGL/ADL/RKL/RPL/DG2)

Install the new packages and run some tests:
- check that the desktop is still using hw acceleration and hasn't fallen back to swrast/llvmpipe
- run freely available benchmarks that torture the GPU (Unigine Heaven/Valley/Superposition)
- run some games from Steam if possible

and in each case check that there is no gfx corruption happening or worse.

Note that upstream releases have already been tested for OpenGL and Vulkan conformance by their CI.

[Where things could go wrong]
This is a major update of Mesa, there could be regressions but we'll try to catch any with testing. And since it shares bugs with mantic, we'd already know if there are serious issues. We will backport the final 23.2.x at a later stage, the first backport is needed for enabling Intel Meteor Lake.

[Timo Aaltonen (tjaalton)]

no need for new rust bits, we didn't migrate to llvm-16

[Steve Langasek (vorlon)]

Hello Timo, or anyone else affected,

Accepted directx-headers into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/directx-headers/1.610.2-2~ubuntu0.22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

The directx-headers part of this is easy because it's just a build-dependency of mesa, but mesa itself is a big delta to a library that affects all users of the graphics stack: unlike the kernel, which has both GA and HWE versions where the user can opt out of the rolling kernel updates on the LTS, there is only one mesa package, so the impact of any regressions is higher.

I see that the current bug description is in line with the previous update, LP: #2019212; and of course, with respect to hardware enablement, this is an SRU we're going to want to take; but I think it would be best if there were a more explicit test case than just "Install the new mesa on various hw". Do we expect this new mesa to be tested on all certified hardware before it's released to -updates? Is there a feedback mechanism for us to *know* if Certification finds regressions on some of the certified platforms? Is any effort made to check for regressions on non-certified hardware?

[Timo Aaltonen (tjaalton)]

Right, it's probably appropriate to finally have a documented wiki-page for Mesa SRU's, so I'll create one.

Upstream does extensive conformance testing on every release, but they do not cover hardware drivers. For that, we should be able to use Intel's Mesa CI which tests branches on many generations of GPU's from them. In order to use it, they need to grant developer access to it and I have a request pending now.

For AMD hardware we can maybe ask them to test the backport on their CI (AIUI it's not public).

In the past we've had the cert team test the backports on a handful of enabled machines (both Intel & AMD), but having the above would greatly improve the coverage and give greater confidence in these SRU's.

[Timo Aaltonen (tjaalton)]

Access to the vendor CI's won't happen anytime soon, so I'll expand the test case for this stack.

[Steve Langasek (vorlon)]

Hello Timo, or anyone else affected,

Accepted mesa into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mesa/23.2.1-1ubuntu3.1~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted mesa (23.2.1-1ubuntu3.1~22.04.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

mutter/42.9-0ubuntu5 (arm64, ppc64el)
vtk9/9.1.0+really9.1.0+dfsg2-3build1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#mesa

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Bin Li (binli)]

After enabled the proposed, updated the mesa packages, it works perfectly on ThinkPad MTL platforms. Thanks!

Before:
$ /usr/lib/nux/unity_support_test -p
OpenGL vendor string: Mesa
OpenGL renderer string: llvmpipe (LLVM 15.0.7, 256 bits)
OpenGL version string: 4.5 (Compatibility Profile) Mesa 23.0.4-0ubuntu1~22.04.1

After:
$ /usr/lib/nux/unity_support_test -p
OpenGL vendor string: Intel
OpenGL renderer string: Mesa Intel(R) Graphics (MTL)
OpenGL version string: 4.6 (Compatibility Profile) Mesa 23.2.1-1ubuntu3.1~22.04.1

[Jeremy Bícha (jbicha)]

All the failing autopkgtests currently reported for this update also failed a migration-reference/0 run

mutter/42.9-0ubuntu5 (arm64, ppc64el)
vtk9/9.1.0+really9.1.0+dfsg2-3build1 (armhf)

[Sergio Costas (rastersoft-gmail)]

Can somebody modify the description to specify exactly how to do those tests, please? (which commands/parameters, and expected results).

[Timo Aaltonen (tjaalton)]

unigine tests take no parameters, just launch and run

[Dimitri John Ledkov (xnox)]

i guess rebuilding gnome snaps with proposed on arm64 and testing that new gnome snap on mantic for pi5 & x1s would help.

[Tim Holmes-Mitra (timhm)]

I've tested the proposed mesa + hwe kernel and as far as I can tell it's working well. I believe my Z16 is navi 2nd generation and it has an iGPU and dGPU both AMD (600M series and RX 6500 respectively). What I tested:

- firefox you tube videos (with both DRI_PRIME=0 and 1)
- steam deb running dota and quake2 (with both DRI_PRIME=0 and 1)
- desktop is hardware rendered with the iGPU and working well
- multi-monitor (2 external 4k monitors)

Let me know if there's anything else I should add or test.

[Marco Trevisan (Treviño) (3v1n0)]

I can test this too if needed, I've a AMD® Ryzen 7 pro 7840u with radeon 780m graphics (phoenix).

[Timo Aaltonen (tjaalton)]

Any testing is helpful, thanks!

I've now ran tests on Intel DG2/Arc, and things work fine at least on 6.5 kernel. 6.1 (OEM) fails to start gdm/X with a trace from the DRI driver, need to investigate if that's a regression or not.

[Anson Tsao (ansontsao)]

Verified this new mesa proposed on my end with oem-6.5, looks positive to launch Firefox with multiple tabs to play videos on YouTube.

[Verification]: HP platform(Under development phase)
AMD R9 PRO 8945HS w/ Radeon 780M Graphics

[David Perry (davidperry)]

Verified on a few different reference boards internally by running a set of 3 Unigine benchmarks with the updated mesa on the following:

Stormpeak W/Radeon 7900 XTX

Stormpeak W/Radeon Pro W7500

Stormpeak W/Radeon Pro W7900

Phoenix AMD Ryzen 7 7840U W/AMD Radeon™ 780M

Phoenix2 AMD Ryzen 5 7545U W/AMD Radeon™ 740M

verified also on a couple different pre-production platforms.

[Timo Aaltonen (tjaalton)]

Thanks a lot for these, so we're good on AMD platforms.

[Timo Aaltonen (tjaalton)]

Ok, I've tested on Intel gen9 (KBL) and gen12 (ADL) so that side is covered too. Gen11 (Ice Lake) is rather rare, so these two generations cover enough.

The issue I had with DG2 on 6.1 OEM kernel was caused by missing a commit which allows reducing the minimum memory alignment. I've reverted the commit from mesa, as that way things work the same as before, and is safest regarding updates. This issue is tracked on bug 2051068. Once the new upload is in proposed, *this* bug can be marked as verified.

[Andreas Hasenack (ahasenack)]

Hello Timo, or anyone else affected,

Accepted mesa into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mesa/23.2.1-1ubuntu3.1~22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

regression fix is in, so this can be marked as verified

[Andreas Hasenack (ahasenack)]

Technically, the verification from prior comments was performed on 23.2.1-1ubuntu3.1~22.04.1, not 23.2.1-1ubuntu3.1~22.04.2. I think they need to be done on the mesa package that is in proposed now, which is .2.

[Łukasz Zemczak (sil2100)]

Timo, can you verify this?

[Timo Aaltonen (tjaalton)]

tested on ADL again, still good

[Launchpad Janitor (janitor)]

This bug was fixed in the package mesa - 23.2.1-1ubuntu3.1~22.04.2

---------------
mesa (23.2.1-1ubuntu3.1~22.04.2) jammy; urgency=medium

  * patches: Revert a patch that caused GUI fail with OEM-6.1 kernel on
    Intel DG2. (LP: #2051068)

mesa (23.2.1-1ubuntu3.1~22.04.1) jammy; urgency=medium

  * Backport to 22.04. (LP: #2037604)
  * rules: Disable rusticl, FTBFS on jammy.

 -- Timo Aaltonen <email address hidden> Wed, 24 Jan 2024 14:01:34 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package directx-headers - 1.610.2-2~ubuntu0.22.04.1

---------------
directx-headers (1.610.2-2~ubuntu0.22.04.1) jammy; urgency=medium

  * Backport to jammy. (LP: #2037604)

 -- Timo Aaltonen <email address hidden> Fri, 27 Oct 2023 08:57:55 +0300

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for mesa has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.