KVM Virtual Machines go into paused a few seconds after starting up

[Márcio Santos]

I have a Testing Virtual Machine Host running with Ubuntu 12.04 LTS Server x64.

It has been running stable since the day after the release of Ubuntu 12.04 when it was installed.

Today i discovered that all the virtual machines were in a paused state.

When i try to resume them with "libvirt resume domain" they startup and go into paused state a few seconds later.

I can't find any errors on libvirt log, domain log or dmesg.

Would appreciate some help debugging and solving the problem.

lsb_release -rd
    Description: Ubuntu 12.04.1 LTS
    Release: 12.04

apt-cache policy qemu-kvm
    qemu-kvm:
      Installed: 1.0+noroms-0ubuntu14.1
      Candidate: 1.0+noroms-0ubuntu14.1
      Version table:
     *** 1.0+noroms-0ubuntu14.1 0
            500 http://archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
            500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages
            100 /var/lib/dpkg/status
         1.0+noroms-0ubuntu13 0
            500 http://archive.ubuntu.com/ubuntu/ precise/main amd64 Packages

The virtual qed file appears to be fine and not being over committed like a few bugs that are going around.

qemu-img info ubuntu-nginx.qed
    image: ubuntu-nginx.qed
    file format: qed
    virtual size: 128G (137438953472 bytes)
    disk size: 7.6G
    cluster_size: 65536

du -sh ubuntu-nginx.qed
    7.6G ubuntu-nginx.qed

Attached follows dmesg, syslog, libvirt log and domain log since last restart.

[Serge Hallyn (serge-hallyn)]

Thanks for reporting this bug.

When I've seen this before, the cause was a full disk on the host. Could you please show the output of 'df -h'?

[Márcio Santos (marcio.santos)]

The drive still has some space available.

df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/sdd1 67G 42G 22G 67% /
    udev 3.9G 4.0K 3.9G 1% /dev
    tmpfs 1.6G 344K 1.6G 1% /run
    none 5.0M 0 5.0M 0% /run/lock
    none 3.9G 76K 3.9G 1% /run/shm
    cgroup 3.9G 0 3.9G 0% /sys/fs/cgroup
    /dev/md0p1 294G 229G 51G 83% /storage
    /dev/sdc1 140G 133G 5.7G 96% /kvm

The images are located on /dev/sdc1, mounted on /kvm.

Is it possible that 5.7GB isn't enough for the hypervisor to work correctly?

Thanks for the help.

[Márcio Santos (marcio.santos)]

Solved...

Your comment about hard disc space made me wonder if maybe it could be some problem with the filesystem on the partition.

A quick sudo e2fsck -f /dev/scd1 found some corrupt inodes that were fixed and the problem went away.

All the virtual machines are running happily again, or so it seems.

Thanks for all your help, Serge.

Kind regards.

[Serge Hallyn (serge-hallyn)]

Thanks for the update!

I'll mark this bug Invalid as it was not a bug in qemu-kvm. Glad it was resolved.