Vulkan not working if nvidia-drm.modeset=1 is set and gdm3 is used (Optimus)

[nastys]

Lenovo ideapad 510-15IKB
OS: Ubuntu 18.04 bionic
Kernel: x86_64 Linux 4.15.0-20-generic
DE: GNOME
CPU: Intel Core i5-7200U @ 4x 3.1GHz
GPU: Intel HD 620 + NVIDIA GeForce 940MX (Optimus)
UEFI boot, Secure Boot disabled.

I installed the proprietary graphics driver with Software Properties and Vulkan applications (e.g. vulkan-smoketest, Unreal Editor, SDK examples, Dolphin) were working fine.
To get rid of tearing I added nvidia-drm.modeset=1 to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, then ran update-grub && update-initramfs -u and rebooted.
After doing that, Vulkan applications stopped working because they couldn't initialize the Vulkan swap chain until I reverted the changes to GRUB.

Everything works as expected if lightdm is used instead of gdm3.

[Daniel van Vugt (vanvugt)]

There's probably no way GDM could cause this directly. But indirectly if you are logging into a Wayland session in one case and Xorg in the other, then that might be the main factor here.

Please try logging into both Xorg and Wayland sessions to see if one works better.

Also, for Nvidia there used to be settings to control tearing in the nvidia control panel. So you shouldn't need to "To get rid of tearing I added nvidia-drm.modeset=1"...?

[nastys (alisfquarto)]

nvidia-drm.modeset=1 is used for PRIME Sync. In a gist, it copies the framebuffer from the dGPU to the iGPU without tearing. Without this, VSync is not effective no matter what.

I have switched back to gdm3. In Xorg, OpenGL works as expected, with the NVIDIA GPU (according to glxinfo), but Vulkan still doesn't work. In Wayland, OpenGL is rendered by the Intel GPU and when I try to run vulkan-smokeinfo I get:

Xlib: extension "NV-GLX" missing on display ":0".
terminate called after throwing an instance of 'std::runtime_error'
  what(): VkResult -3 returned

I remember Ubuntu 17.10 wouldn't let me log in at all if nvidia-drm.modeset=1 was enabled, but lightdm worked without issues (and it still does). Also SSDM is not affected either. Somehow gdm3 doesn't like this option...

[nastys (alisfquarto)]

I have also added WaylandEnabled=false to /etc/gdm3/custom.conf and Vulkan still doesn't work. I also lost the option to log into Wayland from the login screen.

@vanvugt Are you sure this isn't a bug in gdm3?

[Daniel van Vugt (vanvugt)]

"VSync" and "PRIME Sync" are unrelated things. You need to ignore the fact that they sound similar. You should be able to achieve perfect VSync regardless of the "modeset" option. But it may be disabled by default (by Nvidia) as alluded to earlier.

The "NV-GLX" error sounds expected... In a Wayland session you're using Xwayland which doesn't yet connect to the Nvidia driver properly. So avoid Wayland sessions in your case - they're not expected to work for this.

So I think you need to:
  1. Don't use nvidia-drm.modeset=1
  2. Only use Xorg sessions.
  3. Now if tearing is your only problem then look in nvidia-settings for options to fix that. You might also find some xorg.conf options that help here: http://us.download.nvidia.com/XFree86/Linux-x86_64/390.25/README/xconfigoptions.html

gdm3 has nothing to do with any of this. It just launches X and the Nvidia driver dictates what features are supported or not. If you want the maximum features and performance from Nvidia then you usually need to be using a Xorg session and not "modeset".

[nastys (alisfquarto)]

I know that VSync and PRIME Sync are not the same thing. With modeset disabled, VSync IS enabled but there is tearing regardless, and I can see that even just by dragging a window around the screen.

glxgears reports:
Running synchronized to the vertical refresh. The framerate should be
approximately the same as the monitor refresh rate.
37879 frames in 5.0 seconds = 7575.718 FPS
38056 frames in 5.0 seconds = 7611.121 FPS
[...]

There is absolutely no option in nvidia-settings to fix this. I have taken a few screenshots:
https://imgur.com/a/EqRi1NT

Note that there are a lot more options on a desktop computer with a PCI-E NVIDIA graphics card.

This post by an NVIDIA explains why PRIME Synchronization is critical to get rid of tearing on Optimus laptops and why nvidia-drm modeset=1 is necessary:
https://devtalk.nvidia.com/default/topic/957814/linux/prime-and-prime-synchronization/

I quote:
"The NVIDIA driver’s PRIME Synchronization support relies on DRM-KMS, which is disabled by default due to its current incompatibility with SLI. To enable it, run ‘sudo rmmod nvidia-drm; sudo modprobe nvidia-drm modeset=1’. In other words, load the nvidia-drm module with the parameter modeset=1."

Instead of reloading the module every time I need to use my laptop, I just added that parameter to GRUB, which works but breaks Vulkan if gdm3 is used.

If I run:
$ xrandr --output eDP-1-1 --set "PRIME Synchronization" 1
without modeset enabled, the screen goes blank for a couple of seconds and GNOME is reloaded, but I still get tearing and according to xrandr it's still disabled:
$ xrandr --verbose
Screen 0: minimum 8 x 8, current 1920 x 1080, maximum 16384 x 16384
eDP-1-1 connected primary 1920x1080+0+0 (0x47) normal (normal left inverted right x axis y axis) 344mm x 194mm
[...]
 PRIME Synchronization: 0
  supported: 0, 1
[...]

Of course, I get no tearing if modeset is enabled. I don't need to edit my xorg.conf or anything.

Anyway the summary of this bug is "Vulkan not working if nvidia-drm.modeset=1 is set and gdm3 is used (Optimus)", regardless of the tearing issues.

If gdm3 is not affected then why does this ONLY happen with GDM?

[nastys (alisfquarto)]

After some investigating, I have found that if I log out, switch to TTY then run startx, I get the same issue. However I noticed that if I run sudo startx, Vulkan works, and it still works after logging out and running startx as a normal user, until the computer is rebooted. Then I tried logging in as root from GDM and I got the same result.
Something is probably not being initialized properly when logging in as a non-root user.
I think the affected package is actually xinit, but I'm not sure.

[nastys (alisfquarto)]

I have found out just now that if I run anything Vulkan as root (e.g. sudo vulkan-smoketest), IT WORKS! And if I then run it without sudo, IT STILL WORKS!

At this point I have no idea what is going on, nor why this doesn't affect lightdm nor sddm.
It seems the affected package is indeed nvidia-graphics-drivers-390. Since it is proprietary, I'll ask in the NVIDIA Devtalk forums.

[Daniel van Vugt (vanvugt)]

OK. Sounds like there is a device (/dev/* file) that you're not getting correct access to. And you can only get around that limitation by running as root (which is not recommended, for security).

In this case we may find it's an nvidia driver (or installation) bug. Although your original observation that it's a gdm problem might actually explain this too. I don't know if it's gdm, systemd or something else that handles the required privilege escalation to access /dev/GRAPHICS_THINGS so am reopening the gdm task in the hope that somebody else knows...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Mariusz Domański (mario.7)]

I can confirm this is still an issue.

My setup is:
Ubuntu 18.04
MSI GE60
Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz
Intel HD 4600 + GeForce GTX 760M
nvidia-driver-396 (396.54-0ubuntu0~gpu18.04.1 from grphics-drivers PPA)
nvidia-prime for switching cards

With nvidia-drm.modeset=1 until some vulkan app is run as root the vulkaninfo crashes like this:

==========
VULKANINFO
==========

Vulkan Instance Version: 1.1.82

ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_radeon.so: wrong ELF class: ELFCLASS32
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_intel.so: wrong ELF class: ELFCLASS32
INTEL-MESA: warning: Haswell Vulkan support is incomplete
INTEL-MESA: warning: ../../../src/intel/vulkan/anv_device.c:1150: FINISHME: Implement pop-free point clipping

Instance Extensions:
====================
Instance Extensions count = 17
 VK_EXT_acquire_xlib_display : extension revision 1
 VK_EXT_debug_report : extension revision 9
 VK_EXT_debug_utils : extension revision 1
 VK_EXT_direct_mode_display : extension revision 1
 VK_EXT_display_surface_counter : extension revision 1
 VK_KHR_device_group_creation : extension revision 1
 VK_KHR_display : extension revision 23
 VK_KHR_external_fence_capabilities : extension revision 1
 VK_KHR_external_memory_capabilities : extension revision 1
 VK_KHR_external_semaphore_capabilities: extension revision 1
 VK_KHR_get_display_properties2 : extension revision 1
 VK_KHR_get_physical_device_properties2: extension revision 1
 VK_KHR_get_surface_capabilities2 : extension revision 1
 VK_KHR_surface : extension revision 25
 VK_KHR_wayland_surface : extension revision 6
 VK_KHR_xcb_surface : extension revision 6
 VK_KHR_xlib_surface : extension revision 6
Layers: count = 3
=======
VK_LAYER_LUNARG_standard_validation (LunarG Standard Validation Layer) Vulkan version 1.0.82, layer version 1
 Layer Extensions count = 0
 Devices count = 2
  GPU id : 0 (Intel(R) Haswell Mobile)
  Layer-Device Extensions count = 0
  GPU id : 1 (GeForce GTX 760M)
  Layer-Device Extensions count = 0

VK_LAYER_VALVE_steam_overlay_32 (Steam Overlay Layer) Vulkan version 1.1.73, layer version 1
 Layer Extensions count = 0
 Devices count = 2
  GPU id : 0 (Intel(R) Haswell Mobile)
  Layer-Device Extensions count = 0
  GPU id : 1 (GeForce GTX 760M)
  Layer-Device Extensions count = 0

VK_LAYER_VALVE_steam_overlay_64 (Steam Overlay Layer) Vulkan version 1.1.73, layer version 1
 Layer Extensions count = 0
 Devices count = 2
  GPU id : 0 (Intel(R) Haswell Mobile)
  Layer-Device Extensions count = 0
  GPU id : 1 (GeForce GTX 760M)
  Layer-Device Extensions count = 0

Presentable Surfaces:
=====================
GPU id : 0 (Intel(R) Haswell Mobile)
Surface type : VK_KHR_xcb_surface
Formats: count = 2
 B8G8R8A8_SRGB
 B8G8R8A8_UNORM
Present Modes: count = 3
 IMMEDIATE_KHR
 MAILBOX_KHR
 FIFO_KHR

VkSurfaceCapabilitiesKHR:
=========================
 minImageCount = 2
 maxImageCount = 0
 currentEx...

[Luca Osvaldo Mastromatteo (lukycrociato)]

Can be the same issue as
https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers-390/+bug/1800465

Only happening with GDM in ubuntu 18.04 with updated nvidia drivers and ubuntu 18.10

[Mariusz Domański (mario.7)]

As a workaround one needs to run (after each reboot):

sudo vulkaninfo

and then all vulkan apps start to work properly until system is shut down.

[Noctis Bennington (noctis13)]

I have the same fail, since I updated Nvidia version (to 410), and then even returning to 396, Vulkan doesn't work, with vulkaninfo appears in terminal the same text as nastys.

But with "sudo vulkaninfo" Vulkan works.

PD: I have Ubuntu 18.04 with Gnome, and Nvidia's drivers.

[Jaap Buurman (mushoz)]

I have the exact same bug. This one seems to be related: https://bugs.launchpad.net/nvidia-drivers-ubuntu/+bug/1800465

[Mariusz Domański (mario.7)]

The bug is still there on Ubuntu 18.04 with newer drivers and kernel:
nvidia-driver-418
linux-image-4.18.0-16-generic

[Michael Aaron Murphy (mmstick76)]

This still occurs in 19.04 with nvidia-driver-418.

[Daniel van Vugt (vanvugt)]

Sounds like the root cause here is the same as bug 1716857.