Nested-virt)L1 (kvm on kvm)guest panic with parameter “-cpu host” in qemu command line.

Bug #994378 reported by Yongjie Ren
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
QEMU
Fix Released
Undecided
Unassigned

Bug Description

Environment:
------------
Host OS (ia32/ia32e/IA64):ia32e
Guest OS (ia32/ia32e/IA64):ia32e
Guest OS Type (Linux/Windows):Linux
kvm.git Commit:19853301ef3289bda2d5264c1093e74efddaeab9
qemu-kvm Commit:69abebf20280152da8fa7c418a819ae51e862231
Host Kernel Version:3.4.0-rc3
Hardware:WSM-EP, Romley-EP

Bug detailed description:
--------------------------
(KVM on KVM) L1 guest panic when starting the L1 guest with “-cpu host” parameter in qemu command line.

Note:
1. when creating guest with “-cpu qemu64,+vmx”, L1 guest and L2 guest can boot
up.
2. This should be a qemu-kvm bug. using '-cpu host' parameter, the following is the result.
Kvm + qemu-kvm =result
19853301 + 69abebf2 = bad
19853301 + 44755ea3 = good
3. when booting up the guest with the good commit of 19853301 + 44755ea3, you can see some
error info, but nested virt works fine. (L1 and L2 guest can boot up.)
“error: feature "i64" not available in set
error: bad option value [extfeature_edx = i64 xd syscall]”

some logs
[root@vt-snb9 x86_64-softmmu]# ./qemu-system-x86_64 -m 2048 -net nic,model=rtl8139 -net tap,script=/etc/kvm/qemu-ifup -hda /root/nested-kvm.qcow -cpu host
error: feature "i64" not available in set
error: bad option value [extfeature_edx = i64 xd syscall]
error: feature "i64" not available in set
error: bad option value [extfeature_edx = i64 xd syscall]
error: feature "i64" not available in set
error: bad option value [extfeature_edx = i64 syscall xd]
error: feature "i64" not available in set
error: bad option value [extfeature_edx = i64 syscall xd]
VNC server running on `::1:5900'

Reproduce steps:
----------------
1.start up a host with kvm (commit: 19853301)
2.rmmod kvm_intel
3.modprobe kvm_intel nested=1
4.qemu-system-x86_64 -m 2048 -hda L1-kvm.img -cpu host

Current result:
----------------
L1 guest panic.

Expected result:
----------------
L1 guest and L2 guest boot up correctly.

Basic root-causing log:
----------------------

Revision history for this message
Nadav Har'El (nyh) wrote :

Can you please check if you run "qemu-system-x86_64 -m 2048 -hda L1-kvm.img -cpu host" but *without * the nested=1 module option, does L1 also panic, or not?

Revision history for this message
Kashyap Chamarthy (kashyapc) wrote :

Short: I can't reproduce here with L1 guest having has host-passthrough for CPU.

Long:
=====

Version Info:
-------------

On Physical host:
~~~~~~~~~~~~~~~~~
$ uname -r; rpm -q libvirt-daemon-kvm qemu
3.10.0-0.rc2.git1.2.fc20.x86_64
qemu-1.4.2-3.fc19.x86_64
libvirt-daemon-kvm-1.0.5.2-1.fc19.x86_64
libguestfs-1.22.3-1.fc19.x86_64

On L1:
~~~~~~
$ uname -r; rpm -q libvirt-daemon-kvm qemu
3.10.0-0.rc3.git0.2.fc20.x86_64
libvirt-daemon-kvm-1.0.5.1-1.fc19.x86_64
qemu-1.4.2-2.fc19.x86_64
[root@dhcp47-209 ~]#

L1 guest CLI:
-------------
[root@bare-metal ~]# ps -ef | grep qemu
qemu 7281 1 67 04:57 ? 00:00:10 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu host -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

L2 guest CLI:
-------------
[root@regular-guest ~]# ps -ef | grep -i qemu
qemu 1138 1 88 05:18 ? 00:00:07 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nguest-01 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid b47c5cbb-b320-ce9d-c595-4e083b0e465d -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/nguest-01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/nguest-01.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:be:d5:8e,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

A search for string 'error' in logs doesn't turn up anything:
[root@nguest-01 ~]# grep -i error /var/log/boot.log
[root@nguest-01 ~]# grep -i error /var/log/messages
[root@nguest-01 ~]#

Yongjie, can you please re-try?

Revision history for this message
Yongjie Ren (yongjie-ren) wrote :

after "echo 1 > /sys/modules/kvm/parameters/ignore_msrs" in host, the guest (with my own kernel config) boot up fine.

Revision history for this message
chao zhou (chao-zhou) wrote :

when L1 guest kernel: 3.16.0(kvm.git+ qemu.git: c77dcacb..-69f87f71)
create L1 guest:
qemu-system-x86_64 -enable-kvm -m 6G -smp 4 -net nic,macaddr=00:12:31:45:56:13 -net tap,script=/etc/kvm/qemu-ifup ia32e_nested-kvm.img -cpu host
the L1 guest boot up fine

Revision history for this message
chao zhou (chao-zhou) wrote :

this patch fixed the bug:
commit 338b522ca43cfd32d11a370f4203bcd089c6c877
Author: Kan Liang <email address hidden>
Date: Mon Jul 14 12:25:56 2014 -0700

    perf/x86/intel: Protect LBR and extra_regs against KVM lying

    With -cpu host, KVM reports LBR and extra_regs support, if the host has
    support.

    When the guest perf driver tries to access LBR or extra_regs MSR,
    it #GPs all MSR accesses,since KVM doesn't handle LBR and extra_regs support.
    So check the related MSRs access right once at initialization time to avoid
    the error access at runtime.

    For reproducing the issue, please build the kernel with CONFIG_KVM_INTEL = y
    (for host kernel).
    And CONFIG_PARAVIRT = n and CONFIG_KVM_GUEST = n (for guest kernel).
    Start the guest with -cpu host.
    Run perf record with --branch-any or --branch-filter in guest to trigger LBR
    Run perf stat offcore events (E.g. LLC-loads/LLC-load-misses ...) in guest to
    trigger offcore_rsp #GP

    Signed-off-by: Kan Liang <email address hidden>
    Signed-off-by: Peter Zijlstra <email address hidden>
    Cc: Andi Kleen <email address hidden>
    Cc: Arnaldo Carvalho de Melo <email address hidden>
    Cc: Linus Torvalds <email address hidden>
    Cc: Maria Dimakopoulou <email address hidden>
    Cc: Mark Davies <email address hidden>
    Cc: Paul Mackerras <email address hidden>
    Cc: Stephane Eranian <email address hidden>
    Cc: Yan, Zheng <email address hidden>
    Link: http://<email address hidden>
    Signed-off-by: Ingo Molnar <email address hidden>

chao zhou (chao-zhou)
Changed in qemu:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.