kvm virtio block device SEGFAULT

[Adam Guthrie]

Binary package hint: qemu-kvm

I am using opennebula and libvirt/kvm to provision VMs using iSCSI based storage. This works successfully however occasionally VMs disappear (unsure what happens here).

When trying to recreate them (virsh create /path/to/vm.xml) the VM constantly SEGFAULTs whilst booting. I have tried running the kvm command manually with a reduced number of options, and this seems to suggest virtio is at fault. For example:

/usr/bin/kvm -M pc-0.12 -enable-kvm -m 512 -smp 1 -name one-0 -boot c -drive file=/srv/cloud/one/var//0/images/disk.0,if=virtio,index=0,boot=on -vnc 0.0.0.0:10

segfault whereas

/usr/bin/kvm -M pc-0.12 -enable-kvm -m 512 -smp 1 -name one-0 -boot c -drive file=/srv/cloud/one/var//0/images/disk.0,index=0,if=scsi,boot=on -vnc 0.0.0.0:10

boots fine. The stacktrace is:

#0 virtio_blk_rw_complete (opaque=0x1000190, ret=-5) at /build/buildd/qemu-kvm-0.12.3+noroms/hw/virtio-blk.c:129
#1 0x0000000000472f6e in multiwrite_user_cb (mcb=0xcff350) at block.c:1610
#2 0x0000000000473007 in multiwrite_cb (opaque=0xcff350, ret=-5) at block.c:1625
#3 0x000000000047718d in posix_aio_process_queue (opaque=<value optimized out>) at posix-aio-compat.c:460
#4 0x0000000000477288 in posix_aio_read (opaque=0xc781a0) at posix-aio-compat.c:501
#5 0x000000000040c4bf in main_loop_wait (timeout=1000) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:3999
#6 0x0000000000425afa in kvm_main_loop () at /build/buildd/qemu-kvm-0.12.3+noroms/qemu-kvm.c:2122
#7 0x000000000040f046 in main_loop (argc=<value optimized out>, argv=0x7fffffffe658, envp=<value optimized out>)
    at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4210
#8 main (argc=<value optimized out>, argv=0x7fffffffe658, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:6238

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: qemu-kvm 0.12.3+noroms-0ubuntu9.2
ProcVersionSignature: Ubuntu 2.6.32-24.42-server 2.6.32.15+drm33.5
Uname: Linux 2.6.32-24-server x86_64
Architecture: amd64
Date: Wed Sep 8 14:29:42 2010
InstallationMedia: Ubuntu-Server 10.04 LTS "Lucid Lynx" - Release amd64 (20100427)
MachineType: Supermicro X8DTT
ProcCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.32-24-server root=/dev/mapper/os-root ro quiet
ProcEnviron:
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: qemu-kvm
dmi.bios.date: 03/08/2010
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 080016
dmi.board.asset.tag: 1234567890
dmi.board.name: X8DTT
dmi.board.vendor: Supermicro
dmi.board.version: 2.0
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 17
dmi.chassis.vendor: Supermicro
dmi.chassis.version: 1234567890
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr080016:bd03/08/2010:svnSupermicro:pnX8DTT:pvr1234567890:rvnSupermicro:rnX8DTT:rvr2.0:cvnSupermicro:ct17:cvr1234567890:
dmi.product.name: X8DTT
dmi.product.version: 1234567890
dmi.sys.vendor: Supermicro

[Serge Hallyn (serge-hallyn)]

Hi,

thanks for reporting this bug and helping to make Ubuntu better.

I'm still trying to set up a way to reproduce this, but my
first suggestion would be to try removing the '-boot c' from
the command using virtio. That combination has been known
to cause problems. Please let us know if that way boots.

[Serge Hallyn (serge-hallyn)]

Regarding the original segfaults of KVM, it looks like you are having some troubles
with multipath (I see segfaults of multipathd as well as IO errors at least right before
one of the kvm segfaults). I don't know if that is a hardware fault with the controller,
or a bug in the driver that a newer kernel (i.e. from
https://launchpad.net/~kernel-ppa/+archive/pre-proposed) might help.

In the meantime, could you post the result of 'virsh dumpxml VMNAME" for some
VMNAME of a working VM?

[Adam Guthrie (therigu)]

The multipath segfaults are this bug:

https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/488285

and I've patched to fix those.

Attached is the output of virsh-dumpxml.xml for one of my VMs.

I'm going to try reproducing this this afternoon and will try removing the boot option.

[Launchpad Janitor (janitor)]

[Expired for qemu-kvm (Ubuntu) because there has been no activity for 60 days.]