KVM is unstable: virtio bug

Bug #968867 reported by max
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
qemu-kvm (Ubuntu)
Won't Fix
High
Unassigned

Bug Description

Hi,

there are another bug. Simptoms: VM with heavy load stop responding on ssh connection and VNC (through KVM). After I shutdown VM (virsh destroy XX) Newly created machines do not work and produce error on screenshot.

When one VM died others continue work but after it destroyed/shutdown all new machines have the same error message.

Regards,
---
ApportVersion: 2.0-0ubuntu2
Architecture: amd64
DistroRelease: Ubuntu 12.04
InstallationMedia: Ubuntu-Server 11.10 "Oneiric Ocelot" - Release amd64 (20111011)
MachineType: Supermicro H8QG6
NonfreeKernelModules: zfs zcommon znvpair zavl zunicode
Package: qemu-kvm 1.0+noroms-0ubuntu10 [origin: Ubuntu]
PackageArchitecture: amd64
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.2.0-21-generic root=UUID=39e3c519-7ef2-44ea-af9f-60f5328337f2 ro
ProcVersionSignature: Ubuntu 3.2.0-21.34-generic 3.2.13
Tags: precise third-party-packages
Uname: Linux 3.2.0-21-generic x86_64
UnreportableReason: This is not an official Ubuntu package. Please remove any third party package and try again.
UpgradeStatus: Upgraded to precise on 2012-03-15 (18 days ago)
UserGroups: adm admin cdrom dialout kvm libvirtd lpadmin plugdev sambashare
dmi.bios.date: 10/28/2011
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 2.0a
dmi.board.asset.tag: 1234567890
dmi.board.name: H8QG6
dmi.board.vendor: Supermicro
dmi.board.version: 1234567890
dmi.chassis.asset.tag: 1234567890
dmi.chassis.type: 17
dmi.chassis.vendor: Supermicro
dmi.chassis.version: 1234567890
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr2.0a:bd10/28/2011:svnSupermicro:pnH8QG6:pvr1234567890:rvnSupermicro:rnH8QG6:rvr1234567890:cvnSupermicro:ct17:cvr1234567890:
dmi.product.name: H8QG6
dmi.product.version: 1234567890
dmi.sys.vendor: Supermicro

Revision history for this message
max (mikhmv) wrote :
Revision history for this message
max (mikhmv) wrote :

max@s0:/var/lib/one/var$ uname -a
Linux s0 3.2.0-20-generic #33-Ubuntu SMP Tue Mar 27 16:42:26 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

max@s0:/var/lib/one/var$ kvm --version
QEMU emulator version 1.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard

 ps aux| grep kvm | grep one-52
oneadmin 62687 28.9 0.3 25283292 839204 ? Sl 01:31 2:22 /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 20040 -smp 60,sockets=60,cores=1,threads=1 -name one-52 -uuid 30c77a47-85c4-65dd-6c21-9d9b3e66cabe -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-52.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/var/lib/one/var//52/images/disk.0,if=none,id=drive-virtio-disk0,format=raw,cache=writeback -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/one/var//52/images/disk.1,if=none,id=drive-virtio-disk4,format=raw,cache=writeback -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk4,id=virtio-disk4 -fsdev local,security_model=mapped,id=fsdev-fs0,path=/tank/biouml-shared -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=VirtFS,bus=pci.0,addr=0x3 -netdev tap,fd=21,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:00:c0:a8:02:1e,bus=pci.0,addr=0x4 -usb -device usb-mouse,id=input0 -vnc 0.0.0.0:52 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -cpu host

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Please run "apport-collect 968867" if possible.

Do you have any errors in syslog on the host?

If you remove the 9p fs, does this still happen?

Changed in qemu-kvm (Ubuntu):
importance: Undecided → High
status: New → Incomplete
Revision history for this message
max (mikhmv) wrote :

Syslog on host does not contain any errors.
I cannot remove 9p. We store all data on underline FS and sharing between different VM's.

I run apport-collect 968867.
the server doesn't have X. I follow instructions in command line but not sure was it work well.
Does system submit anything?

Regards

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Sorry, no, nothing was added by apport :(

As noone else is experiencing this problem, I suspect it's actually caused by the 9p code, but can't be certain.

I'll need to try to reproduce. If you can try (as mentioned in your other bug) the qemu-kvm package from ppa:ubuntu-virt/ppa, we can see if upstream fixes the issue.

Revision history for this message
max (mikhmv) wrote : BootDmesg.txt

apport information

tags: added: apport-collected precise third-party-packages
description: updated
Revision history for this message
max (mikhmv) wrote : CurrentDmesg.txt

apport information

Revision history for this message
max (mikhmv) wrote : Dependencies.txt

apport information

Revision history for this message
max (mikhmv) wrote : KvmCmdLine.txt

apport information

Revision history for this message
max (mikhmv) wrote : Lspci.txt

apport information

Revision history for this message
max (mikhmv) wrote : Lsusb.txt

apport information

Revision history for this message
max (mikhmv) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
max (mikhmv) wrote : ProcEnviron.txt

apport information

Revision history for this message
max (mikhmv) wrote : ProcInterrupts.txt

apport information

Revision history for this message
max (mikhmv) wrote : ProcModules.txt

apport information

Revision history for this message
max (mikhmv) wrote : RelatedPackageVersions.txt

apport information

Revision history for this message
max (mikhmv) wrote : UdevDb.txt

apport information

Revision history for this message
max (mikhmv) wrote : UdevLog.txt

apport information

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Thanks, the information came through.

can you show the result of 'cat /proc/self/mounts' and 'cat /proc/self/mountinfo'?

Revision history for this message
max (mikhmv) wrote :
Download full text (6.1 KiB)

On host:

max@s0:~$ cat /proc/self/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=132034552k,nr_inodes=33008638,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=52817444k,mode=755 0 0
/dev/disk/by-uuid/39e3c519-7ef2-44ea-af9f-60f5328337f2 / ext4 rw,relatime,errors=remount-ro,user_xattr,acl,barrier=1,data=ordered 0 0
cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sda1 /boot ext4 rw,relatime,user_xattr,acl,barrier=1,data=ordered 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
tank /tank zfs rw,relatime,xattr 0 0
tank/Irina /tank/Irina zfs rw,relatime,xattr 0 0
tank/OpenNebula /tank/OpenNebula zfs rw,relatime,xattr 0 0
tank/biouml-shared /tank/biouml-shared zfs rw,relatime,xattr 0 0

max@s0:~$ cat /proc/self/mountinfo
15 20 0:14 / /sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
16 20 0:3 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
17 20 0:5 / /dev rw,relatime - devtmpfs udev rw,size=132034552k,nr_inodes=33008638,mode=755
18 17 0:11 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
19 20 0:15 / /run rw,nosuid,relatime - tmpfs tmpfs rw,size=52817444k,mode=755
20 1 8:2 / / rw,relatime - ext4 /dev/disk/by-uuid/39e3c519-7ef2-44ea-af9f-60f5328337f2 rw,errors=remount-ro,user_xattr,acl,barrier=1,data=ordered
22 15 0:16 / /sys/fs/cgroup rw,relatime - tmpfs cgroup rw,mode=755
23 22 0:17 / /sys/fs/cgroup/cpuset rw,relatime - cgroup cgroup rw,cpuset
24 22 0:18 / /sys/fs/cgroup/cpu rw,relatime - cgroup cgroup rw,cpu
25 22 0:19 / /sys/fs/cgroup/cpuacct rw,relatime - cgroup cgroup rw,cpuacct
26 22 0:20 / /sys/fs/cgroup/memory rw,relatime - cgroup cgroup rw,memory
27 22 0:21 / /sys/fs/cgroup/devices rw,relatime - cgroup cgroup rw,devices
28 22 0:22 / /sys/fs/cgroup/freezer rw,relatime - cgroup cgroup rw,freezer
29 22 0:23 / /sys/fs/cgroup/blkio rw,relatime - cgroup cgroup rw,blkio
30 22 0:24 / /sys/fs/cgroup/perf_event rw,relatime - cgroup cgroup rw,perf_event
31 15 0:25 / /sys/fs/fuse/connections rw,relatime - fusectl none rw
32 15 0:6 / /sys/kernel/debug rw,relatime - debugfs none rw
33 15 0:10 / /sys/kernel/security rw,relatime - securityfs none rw
34 19 0:26 / /run/lock rw,nosuid,nodev,noexec,relatime - t...

Read more...

Revision history for this message
max (mikhmv) wrote :

Looks like, now VM is stable and works for long time under heavy load without problems.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Thanks, Max. We'll have to find the commits which fixed these and attempt to cherrypick them for an SRU.

Changed in qemu-kvm (Ubuntu):
status: Incomplete → Triaged
tags: added: fixed-upstream
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Ubuntu 12.04 is no longer supported.

Changed in qemu-kvm (Ubuntu):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.