GDM should not allow X11 sessions when NVIDIA's KMS is enabled

[Alberto Milone]

Due to a current limitation in the nvidia binary driver, when enabling KMS (to use Wayland), any X11 sessions will crash.

Ideally, GDM would check if modesetting is enabled in the nvidia driver, and blacklist the X11 sessions.

The following command can tell you whether the parameter is enabled or not:

:$ sudo cat /sys/module/nvidia_drm/parameters/modeset
Y

NOTE: KMS is disabled by default in the nvidia driver, and it is possible to enable it by editing the following file

/etc/modprobe.d/nvidia-graphics-drivers.conf

and by changing the following line:

options nvidia_375_drm modeset=0

to this:

options nvidia_375_drm modeset=1

Then you will probably have to update the initramfs too.

[Doug McMahon (mc3man)]

Don't see it crashing X11 session here at all, 17.10 -
$ echo $DESKTOP_SESSION
ubuntu

sudo cat /sys/module/nvidia_drm/parameters/modeset
Y

(- But the disappointing thing is this -

$ xrandr --verbose
Screen 0: minimum 8 x 8, current 1920 x 1080, maximum 16384 x 16384
eDP1 connected primary 1920x1080+0+0 (0x48) normal (normal left inverted right x axis y axis) 350mm x 190mm
 Identifier: 0x43
 Timestamp: 14688
 Subpixel: unknown
 Gamma: 1.0:1.0:1.0
 Brightness: 1.0
 Clones:
 CRTC: 0
 CRTCs: 0 1 2
 Transform: 1.000000 0.000000 0.000000
             0.000000 1.000000 0.000000
             0.000000 0.000000 1.000000
            filter:
 _MUTTER_PRESENTATION_OUTPUT: 0
 EDID:
  00ffffffffffff0030e4d90200000000
  00150103802313780a15d59e59509826
  0e505400000001010101010101010101
  0101010101017e3680b070381f403020
  350059c2100000190000000000000000
  00000000000000000000000000fe004c
  4720446973706c61790a2020000000fe
  004c503135365746312d544c4232004b
 PRIME Synchronization: 0
  supported: 0, 1

So currently -
no nvidia support in gdm3/wayland
nvidia support in gdm3/xserver-xorg but no PRIME Synchronization support.
i.e., a complete fail in Ubuntu all around

[Doug McMahon (mc3man)]

Actually stand corrected, after a couple of restarts PRIME Synchronization is now working.
So until you all can provide nvidia thru Wayland X11 sessions Should be allowed..

$ xrandr --verboseScreen 0: minimum 8 x 8, current 1920 x 1080, maximum 16384 x 16384
eDP-1-1 connected primary 1920x1080+0+0 (0x46) normal (normal left inverted right x axis y axis) 345mm x 194mm
 Identifier: 0x42
 Timestamp: 18328
 Subpixel: unknown
 Gamma: 1.0:1.0:1.0
 Brightness: 1.0
 Clones:
 CRTC: 0
 CRTCs: 0 1 2
 Transform: 1.000000 0.000000 0.000000
             0.000000 1.000000 0.000000
             0.000000 0.000000 1.000000
            filter:
 _MUTTER_PRESENTATION_OUTPUT: 0
 EDID:
  00ffffffffffff0030e4d90200000000
  00150103802313780a15d59e59509826
  0e505400000001010101010101010101
  0101010101017e3680b070381f403020
  350059c2100000190000000000000000
  00000000000000000000000000fe004c
  4720446973706c61790a2020000000fe
  004c503135365746312d544c4232004b
 PRIME Synchronization: 1
  supported: 0, 1

[Doug McMahon (mc3man)]

Just to reiterate, scorecard here on i7-4700MQ, GeForce GT 755M, nvidia-375.66 drivers, prime profile of nvidia selected.

gnome-wayland session, gdm, KMS = total fail
gnome-wayland session. gdm = total fail

default (ubuntu) session, gdm KMS, = all good, happy days, PRIME Synchronization = yes
default (ubuntu) session, lightdm, KMS = all good, even happier days, PRIME Synchronization = yes

So just the opposite of what is stated here & elsewhere, i.e. Will Cooke -

"While Wayland requires KMS (kernel mode setting) to be enabled, there is
a current limitation in the nvidia driver that causes X11 sessions to
crash with KMS. For this reason I am keeping KMS disabled by default, at
least for now. This is not going to be fixed anytime soon, as it
requires major changes in the driver.

Users who want to try Wayland with nvidia will have to set modeset=1 in
/etc/modprobe.d/nvidia-graphics-drivers.conf, and then update the
initramfs.

When KMS is enabled, GDM will start, but entering the default session
will fail with no error that is visible to the user. Ideally, I think we
would check if KMS is enabled with nvidia, and either show only the
Wayland sessions or maybe simply
default to the Wayland session."

[Iain Lane (laney)]

Doug, are you saying that what Alberto is saying - that X11 sessions with KMS enabled on Nvidia is broken - isn't right?

Maybe the two of you could look into that together.

[Doug McMahon (mc3man)]

On 07/11/2017 12:35 PM, Iain Lane wrote:
> Doug, are you saying that what Alberto is saying - that X11 sessions
> with KMS enabled on Nvidia is broken - isn't right?
>
> Maybe the two of you could look into that together.
>
Absolutely isn't broken, at least here with a hybrid(optimus) device.
To re-confirm just did fresh install from iso of a couple of days ago.
Updated that install fully today.
Installed nvidia drivers (375
rebooted
Enabled KMS via nvidia-graphics-drivers.conf
sudo update-initramfs -u
Rebooted to default session, works fine inc. the added bonus of PRIME
Synchronization: 1 on primary display.

And as mentioned any attempt to log in to a gnome-wayland session fails
with login loop.

[Alberto Milone (albertomilone)]

@Doug: we are going to leave PRIME systems alone in any case. This bug report is about non hybrid systems. You are using the intel GPU to drive the screen, and that is why X11 + KMS works for you. As far as I know, Wayland doesn't support that yet.

[Tim Richardson (tim-richardson)]

What's the bug report for hybrid systems? I can't get gdm working with modeset=1 (aiming to get PRIME sync working). Is good in lightdm. There are so many bug reports with similar symptoms.

[Launchpad Janitor (janitor)]

This bug was fixed in the package gdm3 - 3.25.90.1-0ubuntu4

---------------
gdm3 (3.25.90.1-0ubuntu4) artful; urgency=medium

  [ Jeremy Bicha ]
  * Don't build for s390x since gnome-shell isn't available there

  [ Iain Lane ]
  * We only need to do the above for gdm3 itself. This will make future
    enablement a bit easier; maintaining arch lists is error prone.
  * debian/rules: Only do gdm3 things when we're building gdm3.

 -- Iain Lane <email address hidden> Thu, 07 Sep 2017 16:51:10 +0100