Gnome black screen at boot (QXL + Nvidia hybrid VM)

[Raffaele Rialdi]

I am running Ubuntu 24.04 as host and a VM with Ubuntu 24.04 as guest.
I tried both VirtIO and QXL drivers on Spice (localhost).
The log was created from the VM.

The VMM window is set with these options:
- Scale Display: Always + Auto resize VM with window

Steps to repro:
- I set the resolution of the guest to 1920x1080
- maximize the VMM window (or change the window size in any other way)

The GPU (NVidia 4060 Ti) has been isolated on the host PC and remapped to this VM using iommu.
The remapping worked as I could install the driver and use the GPU libraries for computing purposes.

The problem is that 90% of the boots ends into a black screen right after putting username and password.
I also see the white cross which is typical of the gnome shell not loading.

I also tried to repeatedly switch from Ctl-Alt-F3 (console) to Ctl-Alt-F1 (graphics) to try regaining control of the shell, but it rarely works.

In a first place I need a workaround to get back the display.
The goal is to totaly avoid gnome to use the NVidia card which is mapped in the VM to be used for computation exclusively.

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: xorg 1:7.7+23ubuntu3
ProcVersionSignature: Ubuntu 6.8.0-35.35-generic 6.8.4
Uname: Linux 6.8.0-35-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
.proc.driver.nvidia.capabilities.gpu0: Error: path was not a regular file.
.proc.driver.nvidia.capabilities.mig: Error: path was not a regular file.
.proc.driver.nvidia.gpus.0000.07.00.0: Error: path was not a regular file.
.proc.driver.nvidia.registry: Binary: ""
.proc.driver.nvidia.suspend: suspend hibernate resume
.proc.driver.nvidia.suspend_depth: default modeset uvm
.proc.driver.nvidia.version:
 NVRM version: NVIDIA UNIX x86_64 Kernel Module 535.171.04 Tue Mar 19 20:30:00 UTC 2024
 GCC version:
ApportVersion: 2.28.1-0ubuntu3
Architecture: amd64
BootLog: Error: [Errno 13] Permission denied: '/var/log/boot.log'
CasperMD5CheckResult: pass
Date: Thu Jun 13 09:04:15 2024
DistUpgraded: 2024-05-31 18:30:31,947 DEBUG icon theme changed, re-reading
DistroCodename: noble
DistroVariant: ubuntu
GraphicsCard:
 Red Hat, Inc. QXL paravirtual graphic card [1b36:0100] (rev 05) (prog-if 00 [VGA controller])
   Subsystem: Red Hat, Inc. QEMU Virtual Machine [1af4:1100]
 NVIDIA Corporation AD106 [GeForce RTX 4060 Ti 16GB] [10de:2805] (rev a1) (prog-if 00 [VGA controller])
   Subsystem: ASUSTeK Computer Inc. AD106 [GeForce RTX 4060 Ti 16GB] [1043:891b]
InstallationDate: Installed on 2023-12-12 (184 days ago)
InstallationMedia: Ubuntu 23.10.1 "Mantic Minotaur" - Release amd64 (20231016.1)
Lsusb:
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
 Bus 001 Device 003: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Lsusb-t:
 /: Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 480M
     |__ Port 001: Dev 002, If 0, Class=Human Interface Device, Driver=usbhid, 480M
     |__ Port 004: Dev 003, If 0, Class=Human Interface Device, Driver=usbhid, 480M
 /: Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 5000M
MachineType: QEMU Standard PC (Q35 + ICH9, 2009)
ProcEnviron:
 LANG=en_US.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=linux
 XDG_RUNTIME_DIR=<set>
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-35-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro quiet splash vt.handoff=7
SourcePackage: xorg
Symptom: display
UpgradeStatus: Upgraded to noble on 2024-05-31 (13 days ago)
dmi.bios.date: 04/01/2014
dmi.bios.release: 0.0
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.16.3-debian-1.16.3-2
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-q35-8.0
dmi.modalias: dmi:bvnSeaBIOS:bvr1.16.3-debian-1.16.3-2:bd04/01/2014:br0.0:svnQEMU:pnStandardPC(Q35+ICH9,2009):pvrpc-q35-8.0:cvnQEMU:ct1:cvrpc-q35-8.0:sku:
dmi.product.name: Standard PC (Q35 + ICH9, 2009)
dmi.product.version: pc-q35-8.0
dmi.sys.vendor: QEMU
version.compiz: compiz N/A
version.libdrm2: libdrm2 2.4.120-2build1
version.libgl1-mesa-dri: libgl1-mesa-dri 24.0.5-1ubuntu1
version.libgl1-mesa-glx: libgl1-mesa-glx N/A
version.nvidia-graphics-drivers: nvidia-graphics-drivers-* N/A
version.xserver-xorg-core: xserver-xorg-core 2:21.1.12-1ubuntu1
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev N/A
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:22.0.0-1build1
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.99.917+git20210115-1build1
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:1.0.17-2build1

[Daniel van Vugt (vanvugt)]

I would be very surprised if 'qxl' could act as the primary GPU when the secondary is 'nvidia', but let's see...

In the VM, please switch to a VT, log in and run:

  journalctl -b0 > journal.txt
  journalctl -b-1 > prevjournal.txt

and then attach the resulting text files here.

[Raffaele Rialdi (raffaeler)]

Here is the last journal

[Raffaele Rialdi (raffaeler)]

And here is the previous journal

[Raffaele Rialdi (raffaeler)]

Please note that this configuration worked correctly before upgrading Ubuntu 23.10 to Ubuntu 24.04.
For some reason Ubuntu 24 removed the NVidia proprietary driver version 545, therefore I had to downgrade it to version 535.
Apart from this, there are no other changes.
Thanks

[Raffaele Rialdi (raffaeler)]

This is another journal of the last boot. In this journal you will see the gnome shell loading correctly and working as expected.
I hope this can be helpful to understand the differences and getting closer to the bug.

Thanks