libvirt no longer supports format=host_device

[Dukai Gábor]

Hi!
In Ubuntu Lucid.
After upgrading libvirt to 0.7.5-5ubuntu27.5, none of the kvm domains start. I have raw images on lvm partitions.

virsh # start tserver
error: Failed to start domain tserver
error: monitor socket did not show up.: No such file or directory

/var/lib/log/libvirt/qemu/tserver.log:
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 3072 -smp 4 -name tserver -uuid c04cd76d-029b-7644-13ab-c2d144543ea2 -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/tserver.monitor,server,nowait -monitor chardev:monitor -boot c -drive file=/dev/virtual/tserver,if=ide,index=0,boot=on,format=host_device -net nic,macaddr=52:54:00:0f:5f:c5,vlan=0,model=virtio,name=virtio.0 -net tap,fd=50,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:1 -vga cirrus
libvir: Security Labeling error : error calling aa_change_profile()

/etc/libvirt/qemu/tserver.xml attached.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: libvirt0 0.7.5-5ubuntu27.5
ProcVersionSignature: Ubuntu 2.6.32-25.45-server 2.6.32.21+drm33.7
Uname: Linux 2.6.32-25-server x86_64
Architecture: amd64
Date: Sat Oct 23 13:34:50 2010
InstallationMedia: Ubuntu-Server 10.04.1 LTS "Lucid Lynx" - Release amd64 (20100816.2)
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: libvirt

[Jamie Strandboge (jdstrand)]

Thank you for using Ubuntu and reporting a bug. libvirt sometimes doesn't give very helpful error messages, so I am going to ask you to run a series of commands to help locate the problem.

Can you give the output of the following:
$ cat tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 0

and:
$ cat tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 1

If the second works, as a workaround you can adjust /etc/libvirt/qemu.conf to have:
allow_disk_format_probing = 1

and then restart libvirt with:
$ sudo /etc/init.d/libvirt stop
$ sudo /etc/init.d/libvirt start

[Jamie Strandboge (jdstrand)]

Actually, I see the problem:
$ cat /tmp/tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 0
libvir: Domain Config error : internal error unknown disk format 'host_device' for /dev/virtual/tserver
virt-aa-helper: error: invalid VM definition

$ cat /tmp/tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 1
libvir: Domain Config error : internal error unknown disk format 'host_device' for /dev/virtual/tserver
virt-aa-helper: error: invalid VM definition

If you change tserver.xml to use:
      <driver name='qemu' type='raw'/>

instead of:
      <source dev='/dev/virtual/tserver'/>

It works as expected:
$ cat /tmp/tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 0
virt-aa-helper: warning: path does not exist, skipping file type checks
14:21:55.082: warning : virDomainDiskDefForeachPath:5456 : Ignoring open failure on /dev/virtual/tserver: No such file or directory
virt-aa-helper:
/etc/apparmor.d/libvirt/libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2.files
virt-aa-helper:
  "/var/log/libvirt/**/tserver.log" w,
  "/var/lib/libvirt/**/tserver.monitor" rw,
  "/var/run/libvirt/**/tserver.pid" rwk,
  "/dev/virtual/tserver" rw,

$ cat /tmp/tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 1
virt-aa-helper: warning: path does not exist, skipping file type checks
14:22:11.852: warning : virDomainDiskDefForeachPath:5456 : Ignoring open failure on /dev/virtual/tserver: No such file or directory
virt-aa-helper:
/etc/apparmor.d/libvirt/libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2.files
virt-aa-helper:
  "/var/log/libvirt/**/tserver.log" w,
  "/var/lib/libvirt/**/tserver.monitor" rw,
  "/var/run/libvirt/**/tserver.pid" rwk,
  "/dev/virtual/tserver" rw,

libvirt 0.8.3 from Ubuntu 10.10 doesn't support host_device at all:
$ cat ./tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 0
virt-aa-helper: error: invalid VM definition

$ cat ./tserver.xml | /usr/lib/libvirt/virt-aa-helper -r --uuid libvirt-c04cd76d-029b-7644-13ab-c2d144543ea2 --dryrun -p 1
virt-aa-helper: error: invalid VM definition

[Jamie Strandboge (jdstrand)]

In comment #3, I should have said:
If you change tserver.xml to use:
      <driver name='qemu' type='raw'/>

instead of:
      <driver name='qemu' type='host_device'/>

[Jamie Strandboge (jdstrand)]

I have uploaded a preliminary fix to https://launchpad.net/~ubuntu-security-proposed/+archive/ppa. Please test and give feedback here.

[Dukai Gábor (gdukai)]

Thank you, the <driver name='qemu' type='raw'/> change solved the problem. It works with the latest libvirt (0.7.5-5ubuntu27.6).
It was my mistake then, to use a deprecated option. IMHO it would help a lot if the libvirt documentation were more comprehensive.

[Bart Heinsius (bheinsius)]

The preliminary fix uploaded to https://launchpad.net/~ubuntu-security-proposed/+archive/ppa works for me.

[Jamie Strandboge (jdstrand)]

I have sent the following patch upstream. Once I get feedback on it I will roll it out to Ubuntu.

[Shad Owen (owen-shad)]

I'm not sure if this is any help, if it's related, or at all relevant but I thought I'd throw out something that I noticed. I recently did a clean install on my system and upgraded to Maverick. I have had a heck of a time getting my vm's back up and running. I was able to get them to boot if I interfaced with KVM directly however when I used virt-manager I would consistently get an error, "Boot failed: not a bootable disk" followed by "No bootable device." as if there wasn't a boot device at all.

I found this bug and at random tried changing the "allow_disk_format_probing" to 1 and my vm booted. I tested by disabling the disk format probing and my problem returned. My thought was that maybe the vm wasn't able to read the boot sector of the image. It was just a random guess after I've pounded on this problem for about a week.

I'm not terribly experienced with these things but I'd be happy to offer some logs or whatever I can if it would be of service. You might have to help me out with how to obtain them but I'd love to be able to contribute something. Kudos to all of you work on these things, it's greatly appreciated!

Thanks!

[Jamie Strandboge (jdstrand)]

Shad, your issue is related in that upstream changed how it probes disk. For more details, you can read http://www.ubuntu.com/usn/usn-1008-1. Do your VMs also use 'format=host_device'?

[Jamie Strandboge (jdstrand)]

The format=host_device discussion can be seen here:
https://www.redhat.com/archives/libvir-list/2010-November/msg00276.html

Upstream does not have an interest in fixing this, stated that it was never intended to work, and acknowledges some documentation problems. As such, I am going to mark the maverick and natty tasks as "Won't Fix". I will publish my change for Lucid since the security update broke the previously unintentionally working behavior, but this will not be forward ported to Maverick or later.

[Jamie Strandboge (jdstrand)]

Maverick and later users should transition to 'format=raw' and let QEMU handle the protocol automatically.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 0.7.5-5ubuntu27.7

---------------
libvirt (0.7.5-5ubuntu27.7) lucid-security; urgency=low

  * debian/patches/9902-lp665531.patch: restore use of '<driver name='qemu'
    type='host_device'/>' (LP: #665531)
 -- Jamie Strandboge <email address hidden> Sat, 23 Oct 2010 11:01:42 -0500

[Jamie Strandboge (jdstrand)]

http://www.ubuntu.com/usn/usn-1008-4

[ariel (garcia)]

Uhm... this "fix" for Lucid (and therefore probably the older one in 0.7.5-5ubuntu27.5) did really break a working system!!

We had a running system with Opennebula/KVM/libvirt and after last upgrade
     libvirt 0.7.5-5ubuntu27.3 ----> 0.7.5-5ubuntu27.7
creating virtual machines didn't work anymore for qcow2 images...

I don't discuss if "format=host_device" should be supported or not, but in any case the changes affecting it were IMHO unnecessary in a stable release like Lucid!! I confidently perform updates on a stable release expecting the system to keep working, we were forced to downgrade libvirt to the lucid/release version to get it working again.

With Debian that would not happen ;-)