Qemu 1.5-git gpu clock control doesn`t work after guest reboot

[commiethebeastie]

I run qemu from git with such command:

qemu-system-x86_64 -nodefaults -m 4096 -smp 8,cores=4,threads=2,sockets=1 -cpu 'kvm64' -device usb-mouse -M q35 -vga qxl -no-hpet -boot once=c,menu=on -device vfio-pci,host=02:00.0,x-vga=on \
-enable-kvm -monitor stdio -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 -device virtio-serial -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 -net nic,vlan=0,model=e1000 -net tap,ifname=tap0,script=/etc/guest-ifup -usb -device intel-hda -device hda-duplex \
-drive file='/home/<user>/qemu/win7',if=none,id=drive-virtio-disk0,cache=writeback,aio=native,format=qed,discard=on -device virtio-blk-pci,drive=drive-virtio-disk0,id=virtio-disk \
-drive file='/dev/sr0',if=none,id=drive-ide1-0-0,media=cdrom,snapshot=off,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide1-0-0,id=ide1-0-0 \
-spice port=5930,disable-ticketing

Before guest (Windows 7) reboot, videocard works in 3D mode with full frequency. But after reboot videocard works in 3D only with powersafe frequency. Then I must reboot host for recover gpu clock control.

[commiethebeastie (commiethebeastie)]

Linux localhost 3.8.10-gentoo #3 SMP PREEMPT Wed May 1 19:30:30 MSK 2013 x86_64 AMD FX(tm)-8120 Eight-Core Processor AuthenticAMD GNU/Linux

[Alex Williamson (alex-l-williamson)]

One-shot use is about the state of the art until we implement a way for vfio to do a bus reset on devices. A couple comments on the usage; we're using the x-vga option to vfio-pci and also using -vga qxl. Without specifying a bus for the vfio-pci device this will put qxl vga and vfio vga both on the root complex with no way to switch between them aside from completely disabling the device. This likely means qxl is disabled and effectively the same as booting with -vga none. Second oddity, vfio vga support wasn't added to the kernel until 3.9, how does this qemu command like work on 3.8?

[commiethebeastie (commiethebeastie)]

First I added radeon.ko and fglrx.ko to the blacklist.

Second I ran the

modprobe vfio-pci
echo "0000:02:00.0" > /sys/bus/pci/devices/0000\:02\:00.0/driver/unbind
echo "0000:02:00.1" > /sys/bus/pci/devices/0000\:02\:00.1/driver/unbind
echo "1002 6739" > /sys/bus/pci/drivers/vfio-pci/new_id
echo "1002 aa88" > /sys/bus/pci/drivers/vfio-pci/new_id

And third I ran qemu with the -device vfio-pci,host=02:00.0,x-vga=on options

4-th I installed the Catalyst drivers in windows 7 and change display to HDMI output. But during a reboot the Catalyst driver switches the graphics card to powersave mode.

[commiethebeastie (commiethebeastie)]

With kernel-3.9.0 host system hangs after guest (win 7) poweroff or reboot.

[Alex Williamson (alex-l-williamson)]

Try these:

git://github.com/awilliam/linux-vfio.git vfio-vga-reset
git://github.com/awilliam/qemu-vfio.git vfio-vga-reset

When using both this kernel and this qemu, we'll do a PCI bus reset, which should give you much more consistent behavior both between instances of the guest and at guest reset.

[commiethebeastie (commiethebeastie)]

With VFIO_PCI_VGA, vfio-vga-reset branches and "-vga none -device vfio-pci,host=02:00.0,x-vga=on" host system hangs after guest restarting or turning off.

With no VFIO_PCI_VGA, vfio-vga-reset branches and "-vga none -device vfio-pci,host=02:00.0" catalyst drivers works fine in guest. But after guest restarting the video card is running in powersafe mode.

[commiethebeastie (commiethebeastie)]

With VFIO_PCI_VGA, vfio-vga-reset branches and "-vga none -device vfio-pci,host=02:00.0,x-vga=on" I also received an error:

"qemu-system-x86_64: Attempt to reset PCI bus for VGA support failed (Inappropriate ioctl for device). VGA may not work."

But the drivers were run without problems.

[commiethebeastie (commiethebeastie)]

I installed Windows 8 with VFIO_PCI_VGA, vfio-vga-reset branches and "-vga none -device vfio-pci,host=02:00.0,x-vga=on" I received an error:

"qemu-system-x86_64: Attempt to reset PCI bus for VGA support failed (Inappropriate ioctl for device). VGA may not work." on start.

Then I installed the catalyst drivers. I started the Valley benchmark and the video card is working in normally 3D mode. After rebooting the host system does not hang, but in guest graphics card was in powersafe mode with 100 mhz GPU and 300 mhz memry clocks.

[Alex Williamson (alex-l-williamson)]

Please confirm that you're running the kernel from this branch on the host system:

git://github.com/awilliam/linux-vfio.git vfio-vga-reset

Both host kernel and qemu changes are required. Unfortunately the error code from the ioctl makes it difficult to tell if it isn't available in the kernel or failed, something I need to correct in getting them upstream. These branches only modify the x-vga=on path. Comment #8 indicates using this hangs the host, but comments #9 & #10 says the bus reset call didn't work, did you perhaps boot back into the wrong kernel after the system hang?

From your lspci, 02:00.0 and 02:00.1 are below the 00:0b.0 root port. The properties of 00:0b.0 seem to indicate that 02:00.0 and 02:00.1 should be the only iommu group below this root port, so a bus reset should be available, assuming we're using the correct kernel. In addition to verifying the kernel in use, please attach the output of `find /sys/kernel/iommu_groups/`

[commiethebeastie (commiethebeastie)]

When I use vga none -device vfio-pci,host=02:00.0,x-vga=on with "Linux localhost 3.9.0-rc2 #2 SMP PREEMPT Sat May 4 11:45:12 MSK 2013 x86_64 AMD FX(tm)-8120 Eight-Core Processor AuthenticAMD GNU/Linux" and VFIO_PCI_VGA support. System starting with "qemu-system-x86_64: Attempt to reset PCI bus for VGA support failed (Inappropriate ioctl for device). VGA may not work" warning. And Windows 7 loading with the catalyst drivers. But when I reboot the guest, the host hangs up.

On #8, #9, #10 comments I ran system with vfio-vga-reset branch.

[Alex Williamson (alex-l-williamson)]

The above linux tree is based on 3.9.0, not -rc2, so it appears you're not using the correct kernel.

[commiethebeastie (commiethebeastie)]

Sry. With x-vga=on and "Linux localhost 3.9.0+ #3 SMP PREEMPT Sun May 5 00:58:56 MSK 2013 x86_64 AMD FX(tm)-8120 Eight-Core Processor AuthenticAMD GNU/Linux" it`s work fine.

On Sunday or Monday I will test Geforce gt210 and gt610.

[commiethebeastie (commiethebeastie)]

And bad news too. System hangs after guest poweroff.

[commiethebeastie (commiethebeastie)]

Also I tested nvidia gt210. All works fine: 3D, reboot, poweroff, clocks control, bios initialization.

[Alex Williamson (alex-l-williamson)]

So the result is:

HD6850 - works fully, host hang on guest poweroff
GT210 - works fully, no host issues

Is that correct? Are you attempting to rebind the HD6850 to host drivers after qemu is shutdown, or does the host hang happen prior to where that would be possible? What about killing qemu with a ^C, does it hang the host the same way? If you could run the host in text mode or with a serial or net console so we can see if there are any messages prior to the hang, that would be extremely useful.

[commiethebeastie (commiethebeastie)]

>Are you attempting to rebind the HD6850 to host drivers after qemu is shutdown

No, I did not rebind HD6850 to the host system. System hangs at shutdown guest

>HD6850 - works fully, host hang on guest poweroff

Yes.

[commiethebeastie (commiethebeastie)]

In text mode and on net console there are no errors, host system just freezes after guest poweroff. This may be a hang-up the pcie?

[commiethebeastie (commiethebeastie)]

And all work after killing qemu with ^C.

[Thomas Huth (th-huth)]

Triaging old bug tickets... can you still reproduce this issue with the latest version of QEMU? Or could we close this ticket nowadays?

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]