Starting VMs on qcow2 format with base images in level three fails

Bug #696318 reported by Robert Larsen on 2011-01-01
This bug affects 4 people
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)

Bug Description

Also, for some reason the ownership of the base images changes.

To recreate the problem:

/tmp $ kvm-img create -f qcow2 level1.img 10G
Formatting 'level1.img', fmt=qcow2 size=10737418240 encryption=off cluster_size=0
/tmp $ kvm-img create -f qcow2 -b level1.img level2.img
Formatting 'level2.img', fmt=qcow2 size=10737418240 backing_file='level1.img' encryption=off cluster_size=0
/tmp $ kvm-img create -f qcow2 -b level2.img level3.img
Formatting 'level3.img', fmt=qcow2 size=10737418240 backing_file='level2.img' encryption=off cluster_size=0
/tmp $ grep -E '(<name>)|(<source file)' vm.xml
      <source file='/tmp/level3.img'/>
      <source file='/tmp/ubuntu-10.10-server-amd64.iso'/>
/tmp $ virsh define vm.xml
Domain TestVM defined from vm.xml

/tmp $ virsh start TestVM
error: Failed to start domain TestVM
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/6
qemu: could not open disk image /tmp/level3.img: Permission denied

/tmp $ ls -l *.img
-rw-r--r-- 1 robert robert 262144 2011-01-01 20:37 level1.img
-rw-r--r-- 1 libvirt-qemu kvm 262144 2011-01-01 20:37 level2.img
-rw-r--r-- 1 root root 262144 2011-01-01 20:37 level3.img
/tmp $

If you change the source file in the VM definition to 'level2.img' then the VM starts fine.
This worked perfectly in Ubuntu 10.04.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: libvirt-bin 0.8.3-1ubuntu14
ProcVersionSignature: Ubuntu 2.6.35-24.42-generic
Uname: Linux 2.6.35-24-generic x86_64
NonfreeKernelModules: fglrx
Architecture: amd64
Date: Sat Jan 1 20:32:01 2011
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release amd64 (20100429)
 PATH=(custom, user)
SourcePackage: libvirt

Serge Hallyn (serge-hallyn) wrote :

Thank you for taking the time to file this report and helping to make Ubuntu better.

The information in the description shows that level3.img is owned by root:root, and that the reason for not being able to use level3.img was due to permission being denied. Could you chown libvirt-qemu:kvm /tmp/level*.img, and try again?

Note: since libvirt chowns the files before using them, it actually seems more likely that level2.img or level1.img would be the problem. I can devise one or two sequence of events and experiments which would cause one of the base image files to be inaccessible to libvirt. That might be a real bug worth bringing up upstream.

Changed in libvirt (Ubuntu):
status: New → Incomplete
importance: Undecided → Medium
Serge Hallyn (serge-hallyn) wrote :

Sorry, I'd missed the very start of your description. I see that's what you're actually reporting.

I think this is a valid bug with the dac security driver.

Changed in libvirt (Ubuntu):
status: Incomplete → Confirmed
Serge Hallyn (serge-hallyn) wrote :

Looking at the libvirt source code, the chowning of the files to root:root rather than your own credentials is not a bug, but a not-yet-implemented feature (marked by the comment '/* XXX record previous ownership */').

Nevertheless, I assume (hope) that level1.img and level2.img do not get chowned to root:root when level3.img was used as the disk image. So long as that is the case, as a workingaround you should be able to initially chown all three files to libvirt-qemu:qemu. After each session is closed libvirt will chown level3.img to root:root, but not touch the other two. On next startup, level1 and level2 are owned by the right user, and level3 will get chowned.

Please reply and let us know if that is not the case.

Serge Hallyn (serge-hallyn) wrote :

A-ha! It's not the dac driver. It's the apparmor driver. 'grep apparmor /var/log/syslog | tail' gives me a bunch of:

Jan 3 19:04:06 localhost kernel: [11904.438804] type=1400 audit(1294103046.071:33): apparmor="DENIED" operation="open" parent=1 profile="libvirt-e58d045d-d4ed-39eb-09d2-c884173ff64c" name="/tmp/level1.img" pid=15084 comm="kvm" requested_mask="r" denied_mask="r" fsuid=117 ouid=117

The apparmor libvirt driver isn't writing policy to allow access to level1.img.

tags: added: apparmor
Jamie Strandboge (jdstrand) wrote :

This is probably a bug in virDomainDiskDefForeachPath(), as used by get_files() in virt-aa-helper.c and SELinuxSetSecurityImageLabel() in security_selinux.c. virDomainDiskDefForeachPath() is used to enumerate the disk images to add to the dynamic policy for the security drivers.

Jamie Strandboge (jdstrand) wrote :

A workaround is to add the missing path to /etc/apparmor.d/libvirt/libvirt-<uuid> for the VM in question (but not the dynamically generated /etc/apparmor.d/libvirt/libvirt-<uuid>.files). Obviously this has to be down manually on a per machine basis.

zzfancy (zzfancy) wrote :

This issue still exist at ubuntu12.04...
@Jamie Strandboge (jdstrand)
what should i to add in /etc/apparmor.d/libivrt/libvirt-<uuid>.files ?
thx a lot ~

My workarround is to add this to the end of /etc/apparmor.d/abstractions/libvirt-qemu

# Work around for bug 696318 in KVM
/var/lib/libvirt/images/** r,

It works for me on ubuntu precise 12.04

information type: Public → Public Security

workarount from comment #9 works for me too.
Thanks David :)

ubuntu 13.10

information type: Public Security → Public

#9 works perfectly but will this ever be pushed into the distro?
I am adding it manually and have to do so continually as new updates arrive that update the file.

Serge Hallyn (serge-hallyn) wrote :

The approach in comment #9 is not acceptable, as it allows all VM guests to access all other VM guests's data. It also would not suffice. For instance the example in the Description uses /tmp rather than /var/lib/libvirt/images, and would not be fixed.

What we need is for virt-aa-helper to be extended to calculate the files to which access is needed.

If someone is familier with the best way for libvirt code to determine the full chain of backing stores for a particular file, then the patch should be quite simple.

Serge Hallyn (serge-hallyn) wrote :

As it turns out, this is actually a user error.

When creating the level2.img and level3.img, you must specify the backing store type, because libvirt won't automatically detect those for safety reasons. So you should do:

qemu-img create -f qcow2 -o backing_fmt=qcow2 -b level1.img level2.img
qemu-img create -f qcow2 -o backing_fmt=qcow2 -b level2.img level3.img

After doing that, boot succeeds on my laptop.

Changed in libvirt (Ubuntu):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers