Bad interactions with kvm and vmware (CPU left in VMX root mode?)

[xtknight]

Binary package hint: kvm

This happens with the supplied Hardy kvm and the latest manually compiled kvm-66 kernel modules.

I can not boot my VMware virtual machine after the kvm and kvm_intel modules have been loaded. The system hard-crashes (Magic SysRq+B does not reboot the PC).

If I first start the VM and load the kvm and kvm_intel modules, it seems to work for a while. If I then do "kvm_intel" I get two capslock/scrolllock and SysRq+B also does not work here.

I can not get ANY logs from this using the standard procedures. They simply don't work. Nothing is written to the log.

I believe this happens because the CPU is left in VMX root mode. VirtualBox actually tells me to remove kvm and I suspect the problem is that VMware does not handle this exception, instead continuing and hard locking. But the root of the problem (no pun intended, of course!) happens to be in KVM leaving the CPU in VMX root mode.(?) http://www.virtualbox.org/ticket/317

This is a terrible crash (upon reboot one time the 2nd port of my graphics card died until a frozen reboot). Also causes RAID reconstruction and data loss in home directory. SysRq+B does not even reboot the PC! Very difficult to debug, if not impossible with the stock kernel. Something needs to be done about this, in my opinion, because a lot of people who use VMware also want to experiment with kvm/qemu.

Oh, it also has the potential of destroying your virtual machine. The VMX file will almost certainly be wiped completely and the .vmem file has the potential of being corrupted. You'll be lucky if no data loss ensues.

Linux andy-desktop 2.6.24-16-generic #1 SMP Thu Apr 10 12:47:45 UTC 2008 x86_64 GNU/Linux

I am pretty sure I was able to reproduce this on a Hardy LiveCD as well.

From time to time I get this in dmesg, I don't know if it's related or not. Even without kvm loaded...:

[ 97.914251] VMBlock warning: DentryOpRevalidate: invalid args from kernel
[ 97.914460] VMBlock warning: DentryOpRevalidate: invalid args from kernel
[ 98.004492] VMBlock warning: DentryOpRevalidate: invalid args from kernel
[ 98.004675] VMBlock warning: DentryOpRevalidate: invalid args from kernel
[ 98.007695] VMBlock warning: DentryOpRevalidate: invalid args from kernel
[ 98.007842] VMBlock warning: DentryOpRevalidate: invalid args from kernel

[xtknight (xt-knight)]

Version Info:

Ubuntu Hardy, kernel linux-image-2.6.24-16-generic 2.6.24-16.30 (also happens with fresh 2.6.25 kernel I believe)
VMware Player 2.0.3 build-80004
kvm-66 (latest upstream) module, also happens with stock Hardy kvm in same fashion

$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz
stepping : 6
cpu MHz : 1866.666
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips : 3736.06
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz
stepping : 6
cpu MHz : 1866.666
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips : 3733.36
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

[ts (tsu-dsl)]

I've got exactly the same problem.

Machine setup is Kubuntu Hardy 8.04 default install, VMplayer 2.0.3-80004 default install.
(kernel 2.6.24-16-generic x86_64 GNU/Linux; machine has Intel E6750, 5GB RAM and Nvidia 6300 graphics card.)

Kernel module kvm_intel loads at boot time. Starting VMplayer runs it for a bit and then the machine freezes and mouse and keyboard lock up. The only way to recover is a hard reset.

I manually disabled the kvm boot script to avoid loading the kvm_intel module, and VMplayer worked ok.

Loading kvm_intel manually and starting VMplayer reliably freezes the machine.

The last log entry in /var/log/messages is:

 May 3 21:32:24 xxxx kernel: [ 5233.637595] VMBlock warning: DentryOpRevalidate: invalid args from kernel

It may or may not be related, but googling for it got me to this buglist.

The impact is serious since VMplayer's DirectX 9.0c support in Windows works quite well and KVM (nice for all sorts of experiments and specialized virtual machines with Linux) cannot co-exist with this bug present.

Just one bug away from the ideal world...(?)

[Rocko (rockorequin)]

Confirmed on my system with Hardy 8.04 on a T9300 Intel core duo CPU, kernel 2.6.24-17-generic (64 bit), KVM 1:62+dfsg-0ubuntu7, vmware-server 1.05 build-80187.

Ubuntu freezes totally - you can't even ssh in from another PC.

I'd say the bugfix should be high priority because of its nature (ie it freezes the entire machine, requiring a hard reset and causing data loss for any open files and potentially data corruption for files being written - Linux isn't supposed to ever do this!).

[cement_head (andorjkiss)]

This sounds EXACTLY life the problem I had in Hardy 8.04.1 . On an HP 6910p laptop with ATI video controller. Did a fresh vanilla install. Installed VMWare 1.0.6 such that I could put windows XP on machine as a guest. The initial install worked well and didn't seem to be a problem. I must have installed something either in the VM Guest (XP) or on the Host Ubuntu, because the kernel panic was reproducible and constant. Five minutes after booting the guest OS, hard machine lock up. Causes me to go back to Gutsy. Maybe I'll tool around in the forums again to see if I find a solution.

[tom.delusinne (tom-delusinne)]

I can confirm this bug on my Dell Latitude D830 with Intel T7700 with Ubuntu 8.04.1 running a 32 bit 2.6.24-19-generic kernel. I have kvm and VMware Server (1.0.6 build-91891) installed.

Ubuntu totally freezes up without any logging when starting up a Solaris 10 image in VMware Server.

When I stop kvm manually and then startup the same VMware image, there is no problem at all.
This kvm bug is easily reproducible.

[c.s.prakash (thebase0)]

I got the same problem in Ubuntu 8.04.1 amd64 on MacBook4,1. after removing the kvm module it works fine.

[gmurre (gmurre)]

I had the same problem, I have disabled virtualization in the BIOS and then VMware is working properly.

[Yeik (yeik)]

i had the same problem, but the problem didn't show up until i rebooted after installing KVM.
I am using an intel q6600 6 gb ram ubunutu 8.04 64 bit, VMware server 1,07. as soon as i start a VM in VMware, mouse and keyboard freezes, 1 or 2 CPU's go to 100% and stay there, network connectivity is immediately lost but system monitor still updates for several minutes before that freezes as well. Maybe this is more of a bug for VMWare, but i thought i would post here because it happens to break when i installed kvm, and is working after kvm is removed.