[NVIDIA][Wayland] graphic target not able to reach as the only monitor connect to NV GPU only on I+N machine

[jeremyszu]

[Steps to reproduce]
1. Have an I+N desktop.
2. Attach monitor to dGPU (nvidia)
3. install jammy daily build with enable third-party (to install nvidia-510)
4. dist-upgrade
5. reboot the system and make sure the system starts with Wayland

[Expected result]
System launch gdm login screen without problem

[Actual result]
System stuck with blackscreen

[Other information]
$ systemctl list-jobs
JOB UNIT TYPE STATE
140 system-getty.slice start waiting
2 multi-user.target start waiting
1 graphical.target start waiting
165 plymouth-quit-wait.service start running
8 setvtrgb.service start waiting
110 systemd-update-utmp-runlevel.service start waiting

6 jobs listed.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: gdm3 41.3-1ubuntu2 [modified: lib/udev/rules.d/61-gdm.rules]
ProcVersionSignature: Ubuntu 5.15.0-18.18-generic 5.15.12
Uname: Linux 5.15.0-18-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu78
Architecture: amd64
CasperMD5CheckResult: pass
Date: Sat Mar 5 00:15:24 2022
InstallationDate: Installed on 2022-01-20 (43 days ago)
InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220109)
SourcePackage: gdm3
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile..etc.gdm3.custom.conf: 2022-03-04T20:36:36.997559

[Daniel van Vugt (vanvugt)]

While bug 1959888 is still open we should track it there.

[jeremyszu (os369510)]

Hi Daniel,

This issue is talking about I+N desktop.
It may not same as a hybrid laptop.

[Daniel van Vugt (vanvugt)]

The only difference is which GPU mutter/gnome-shell is trying to use as "primary". According to the log in comment #2 it sounds like Nvidia is indeed the primary here:

 ä¸‰ 04 23:54:57 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Added device '/dev/dri/card1' (nvidia-drm) using non-atomic mode setting.
 ä¸‰ 04 23:54:57 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Device '/dev/dri/card0' prefers shadow buffer
 ä¸‰ 04 23:54:57 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Added device '/dev/dri/card0' (i915) using atomic mode setting.
...
 ä¸‰ 04 23:54:58 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Created gbm renderer for '/dev/dri/card1'
 ä¸‰ 04 23:54:58 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Created gbm renderer for '/dev/dri/card0'
 ä¸‰ 04 23:54:58 ubuntu-ThinkStation-P350-Tower gnome-shell[1127]: Boot VGA GPU /dev/dri/card1 selected as primary

So yeah that's different to most hybrid laptops. Still I would keep the 'hybrid' tag since that word can just mean two GPUs.

[Sebastien Bacher (seb128)]

Daniel, could you check what we need here?

[jeremyszu (os369510)]

I did a review on the upstream MP that Daniel shared.
It looks to me that the upstream intend to disable Wayland in many of scenarios.
It contains a case:
(proprietary nvidia) + (hybrid graphics) + (laptop)

It make me feel curious that why only impact "hybrid laptop" but "hybrid desktop/workstation".

In the other hand, that patch is also what Ubuntu needs to aware.
It refers to https://download.nvidia.com/XFree86/Linux-x86_64/460.39/README/powermanagement.html for something like nvidia-hibernate (Wayland will be disabled if we doesn't enable them)

[Daniel van Vugt (vanvugt)]

In a hybrid (usually muxless) laptop the BIOS will flag the Intel GPU is the boot GPU, which means gnome-shell/mutter will flag that one as "primary" and will do ALL shell rendering on it.

In a hybrid desktop/workstation the BIOS will *usually* disable the integrated GPU leaving only the discrete GPU. But not always, and sometimes it's configurable like in my NUC Extreme. If the BIOS has left both GPUs enabled and booted with only the discrete GPU attached to a monitor then it is likely gnome-shell/mutter will flag that one as "primary" and will do the shell rendering on that "primary" discrete GPU.

"Secondary" GPUs in mutter still do a little work but only so much as copying the work of the primary GPU to external connectors owned by the secondary GPU.

So that explains the difference between a typical hybrid laptop and hybrid desktops.

I think we'll end up with multiple nvidia-wayland bugs in this area and as soon as I have time I will be focussing on bug 1959888 first because upstream (Jonas) seems confident that it's not implemented for hybrid laptops and shouldn't work in Wayland sessions.

[Daniel van Vugt (vanvugt)]

That all said, I haven't seen any reason why we should do anything other than go with upstream's gdm patches. If Wayland needs to be avoided in some Nvidia setups then we should do so to get at least the login screen.

I'm confused though, because some people have made comments suggesting it works. And I'm also wondering if I can implement some of the missing code (maybe) since I've been working on that area of mutter recently.

[Daniel van Vugt (vanvugt)]

The OEM team seem to have this under control and I'm not planning on doing any gdm3 work at all. But as a team we might want to keep an eye on https://gitlab.gnome.org/GNOME/gdm/-/merge_requests/171 and revisit our defaults in the coming weeks.

[jeremyszu (os369510)]

Hi Daniel,

yes, in hybrid system, different BIOS have their own way to configure it. We just need to make sure the memory point of efifb passed from BIOS is correct one and it's correct in this case, the boot_vga is dGPU.

I'll risk this issue on upstream to discuss and share the upstream bug here.

[jeremyszu (os369510)]

Still could reproduce this issue with Jammy daily build (3/13)

$ sha256sum ~/Downloads/jammy-desktop-amd64.iso
1c46933bf1837e6427f994676c60d54515c6ce011276dbfe4c5a63e427fdc2c5 /home/jeremysu/Downloads/jammy-desktop-amd64.iso

1. Install third-part drivers during the installation.
2. After installation, don't choose a mode specifically, it'll use X because of u-d-c patch.
3. sudo apt dist-upgrade
4. reboot with choosing to Wayland.
5. there is no output.

[jeremyszu (os369510)]

In daily build:
kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 470.103.01

From comment#2:
kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 510.39.01

In latest ubuntu archive:
kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 510.54

This issue can be fixed if executing:
sudo ubuntu-drivers install nvidia:510

Thus, the issue is be fixed by latest nvidia driver.
We need to see whether the u-d-c didn't choose 510 in this case.
(I did connect to internet during the instalaltion)

$ ubuntu-drivers list
WARNING:root:_pkg_get_support nvidia-driver-510: package has invalid Support PBheader, cannot determine support level
WARNING:root:_pkg_get_support nvidia-driver-510-server: package has invalid Support PBheader, cannot determine support level
nvidia-driver-470-server, (kernel modules provided by linux-modules-nvidia-470-server-generic-hwe-20.04)
nvidia-driver-510, (kernel modules provided by linux-modules-nvidia-510-generic-hwe-20.04)
nvidia-driver-470, (kernel modules provided by linux-modules-nvidia-470-generic-hwe-20.04)
nvidia-driver-510-server, (kernel modules provided by linux-modules-nvidia-510-server-generic-hwe-20.04)

[jeremyszu (os369510)]

Mar 14 02:38:53 ubuntu ubiquity: WARNING:root:_pkg_get_support nvidia-driver-510: package has invalid Support PBheader, cannot determine support level
Mar 14 02:38:53 ubuntu ubiquity: WARNING:root:_pkg_get_support nvidia-driver-510-server: package has invalid Support PBheader, cannot determine support level
Mar 14 02:38:53 ubuntu ubiquity: Reading package lists...
Mar 14 02:38:53 ubuntu ubiquity:
Mar 14 02:38:53 ubuntu ubiquity: Building dependency tree...
Mar 14 02:38:53 ubuntu ubiquity:
Mar 14 02:38:53 ubuntu ubiquity: Reading state information...
Mar 14 02:38:53 ubuntu ubiquity:
Mar 14 02:38:53 ubuntu ubiquity: The following additional packages will be installed:
Mar 14 02:38:53 ubuntu ubiquity: binutils binutils-common binutils-x86-64-linux-gnu libbinutils libctf-nobfd0
Mar 14 02:38:53 ubuntu ubiquity: libctf0 libnvidia-cfg1-470 libnvidia-common-470 libnvidia-compute-470

...

It looks like detected the nvidia-510 but choosing 470.

[jeremyszu (os369510)]

It seems a regression since https://launchpad.net/ubuntu/+source/nvidia-graphics-drivers-510/510.54-0ubuntu0.18.04.1

because of
```
  * debian/templates/control.in:
    - Set XB-Support to PB (production branch).
```

Checking with maintainer on https://chat.canonical.com/canonical/pl/5bheikgti78qixw1jn9yjq16bc

[jeremyszu (os369510)]

Report a bug as https://bugs.launchpad.net/oem-priority/+bug/1964747

[jeremyszu (os369510)]

This issue has been fixed in 510.54.

For installed incorrect version, please see comment#16.