[MIR] nvidia-modprobe

[Graham Inggs]

This package is needed as a dependency of nvidia-graphics-drivers-* and of nvidia-graphics-drivers-*-updates (main), so that applications making use of CUDA or Nvidia's OpenCL ICD can load the nvidia-uvm driver without requiring root access.

Report:
https://wiki.ubuntu.com/MainInclusionReportNvidiaModprobe

[Michael Terry (mterry)]

Setuid binary that loads a kernel module... Security team will definitely have to weigh in on this. :)

[Graham Inggs (ginggs)]

See LP: #1361207

[Seth Arnold (seth-arnold)]

I'm surprised /etc/modules and udev rules aren't sufficient. Why don't the standard mechanisms for loading modules and creating device nodes work for nvidia drivers?

Thanks

[Graham Inggs (ginggs)]

nvidia-modprobe creates the /dev/nvidia-uvm device node and loads the nvidia_uvm module for a normal user, on demand.

Using clinfo as an example since it is in the archive, small, and wasn't compiled against anything Nvidia (it is built against ocl-icd-libopencl1). See LP: #1499996 for the relationship between ocl-icd-libopencl1 (an OpenCL ICD loader) and nvidia-opencl-icd-352 (an OpenCL ICD).

Without nvidia-modprobe installed, clinfo reports no devices:

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 17:41 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 17:41 /dev/nvidiactl

$ lsmod | grep nvidia
nvidia 10563584 41
drm 356352 3 nvidia

$ clinfo
Number of platforms 0

Without nvidia-modprobe installed, but running as root, clinfo finds the device:

$ sudo clinfo
Number of platforms 1
Platform Name NVIDIA CUDA
<snip>

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 17:41 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 17:41 /dev/nvidiactl
crw-rw-rw- 1 root root 247, 0 Oct 12 17:45 /dev/nvidia-uvm

$ lsmod | grep nvidia
nvidia_uvm 36864 0
nvidia 10563584 42 nvidia_uvm
drm 356352 3 nvidia

So the Nvidia OpenCL ICD is also capable of creating the device node and loading the nvidia_uvm module, provided it is run as root on its first run.

With nvidia-modprobe installed, it "just works" for the user:

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 16:59 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 16:59 /dev/nvidiactl

$ lsmod | grep nvidia
nvidia 10563584 41
drm 356352 3 nvidia

$ clinfo
Number of platforms 1
Platform Name NVIDIA CUDA
<snip>

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 16:59 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 16:59 /dev/nvidiactl
crw-rw-rw- 1 root root 247, 0 Oct 12 17:38 /dev/nvidia-uvm

$ lsmod | grep nvidia
nvidia_uvm 36864 0
nvidia 10563584 42 nvidia_uvm
drm 356352 3 nvidia

Running strace on clinfo shows that the Nvidia OpenCL ICD does the following:
- check if the nvidia_uvm module is loaded, if not, run '/sbin/modprobe' if we are root, otherwise run '/usr/bin/nvidia-modprobe'.
- check if /dev/nvidia-uvm exists, if it does not, try to create it, and if that fails run '/usr/bin/nvidia-modprobe' again.

The same is true of CUDA applications that are linked to libcuda or libcudart.

[Graham Inggs (ginggs)]

Alberto has already written udev rules, and they work, however, as far I can tell, they still require the user to 'sudo modprobe nvidia_uvm' before running any OpenCL or CUDA applications, or manually add nvidia_uvm to /etc/modules.

So the question is, do we want to try to load nvidia_uvm and create /dev/nvidia-uvm on demand, or should it just be done for all nvidia devices?

For desktops, nvidia_uvm could be loaded when nvidia is loaded, but for servers nvidia_uvm would need to be added to /etc/modules.

[Graham Inggs (ginggs)]

Just for reference, the nvidia_uvm module does not seem to be needed for normal desktop and gaming use.
With nvidia-modprobe installed, I opened Steam and the game Portal 2. Afterwards, nvidia_uvm had not been loaded, and /dev/nvidia-uvm had not been created.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Seth Arnold (seth-arnold)]

Graham, thanks for investigating and reporting back. If you're up for some legwork, I'm curious what SUSE and Red Hat do here; I don't expect them to want to use a setuid binary executable for this either and I wonder if they've got the module loading done differently or better.

Thanks

[Graham Inggs (ginggs)]

I can certainly have a look what SUSE do.

[Graham Inggs (ginggs)]

I've tried installing the NVIDIA drivers from the SUSE repository not the NVIDIA website in SLED 12.

nvidia-modprobe is installed, but without setuid being set:

~> which nvidia-modprobe
/usr/bin/nvidia-modprobe

~> ls -l /usr/bin/nvidia-modprobe
-rwxr-xr-x 1 root root 25432 Sep 3 10:57 /usr/bin/nvidia-modprobe

The devices are created and the modules are loaded on startup, not on demand:

~> ls -l /dev/nv*
crw-rw----+ 1 root video 195, 0 Nov 17 11:43 /dev/nvidia0
crw-rw----+ 1 root video 195, 255 Nov 17 11:43 /dev/nvidiactl
crw-rw----+ 1 root video 247, 0 Nov 17 11:43 /dev/nvidia-uvm
crw------- 1 root root 10, 144 Nov 17 11:43 /dev/nvram

~> lsmod | grep nvidia
nvidia_uvm 39162 0
nvidia 10573501 40 nvidia_uvm
drm 322623 2 nvidia

[Graham Inggs (ginggs)]

They have the following in /etc/modeprobe.d/50-nvidia.conf:

options nvidia NVreg_DeviceFileUID=0 NVreg_DeviceFileGID=33 NVreg_DeviceFileMode=0660
install nvidia PATH=$PATH:/bin:/usr/bin; /sbin/modprobe --ignore-install nvidia; /sbin/modprobe nvidia_uvm; test -c /dev/nvidia-uvm || mknod -m 660 /dev/nvidia-uvm c $(cat /proc/devices | while read major device; do if [ "$device" == "nvidia-uvm" ]; then echo $major; break; fi ; done) 0 && chown :video /dev/nvidia-uvm; test -c /dev/nvidiactl || mknod -m 660 /dev/nvidiactl c 195 255 && chown :video /dev/nvidiactl; devid=-1; for dev in $(ls -d /sys/bus/pci/devices/*); do vendorid=$(cat $dev/vendor); if [ "$vendorid" == "0x10de" ]; then class=$(cat $dev/class); classid=${class%%00}; if [ "$classid" == "0x0300" -o "$classid" == "0x0302" ]; then devid=$((devid+1)); test -c /dev/nvidia${devid} || mknod -m 660 /dev/nvidia${devid} c 195 ${devid} && chown :video /dev/nvidia${devid}; fi; fi; done

[Alberto Milone (albertomilone)]

It certainly makes sense to simply set up things when the nvidia module is loaded, although I plan to do it with a udev rule. This will make nvidia-modprobe redundant on the desktop (i.e. where X is started by default).

On the server (and on the desktop with Mir), however, X is not started, and there is nothing that loads the nvidia module (with root privileges). A udev rule should probably try to load the driver when the hardware is available.

This should make nvidia-modprobe largely redundant in both use cases.

[Graham Inggs (ginggs)]

Marking this 'Won't Fix', we can track the udev rule in LP: #1361207.

[Tyler Hicks (tyhicks)]

Thanks for the update! Unassigning and unsubscribing the Ubuntu Security Team based on comment #13.

[Seth Arnold (seth-arnold)]

Graham, thanks for doing the legwork to confirm that a better solution is possible.

Alberto, thanks for making the better solution a reality.

Good work all around :)