setgid, setuid needed by /etc/apparmor.d/abstractions/libvirt-qemu

[s450r1]

IMPACT: libvir cannot setuid to run VMs as non-root
REGRESSION POTENTIAL: there should be none, we are only allowing libvirt to setuid and setgid, not changing any code
TEST CASE:

I couldn't boot any guest VMs with virsh until I modified /etc/apparmor.d/abstractions/libvirt-qemu:
jad@kvmhost:~$ sudo bzr diff /etc/apparmor.d/
=== modified file 'apparmor.d/abstractions/libvirt-qemu'
--- apparmor.d/abstractions/libvirt-qemu 2010-04-30 15:33:20 +0000
+++ apparmor.d/abstractions/libvirt-qemu 2010-05-12 17:26:56 +0000
@@ -8,6 +8,8 @@
   capability dac_override,
   capability dac_read_search,
   capability chown,
+ capability setgid,
+ capability setuid,

   # this is needed with libcap-ng support, however it breaks a lot of things
   # atm, so just silence the denial until libcap-ng works right. LP: #522845

... and restarted apparmor and libvirtd.

Without `capability setgid`, the qemu guest log file contained:
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_
AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.11 -enable-kvm -m 512 -smp 1 -name dm1 -u
uid 79d03a71-3be6-19df-1070-791239480888 -chardev socket,id=monitor,path=/var/li
b/libvirt/qemu/dm1.monitor,server,nowait -monitor chardev:monitor -boot c -drive
 file=/var/vm/dm1/disk0.qcow2,if=ide,index=0,boot=on -drive file=/var/vm/dm1/disk1.qcow2,if=ide,index=1 -net nic,macaddr=52:54:00:bf:75:90,vlan=0,model=virtio,name=virtio.0 -net tap,fd=50,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus
libvir: QEMU error : cannot change to '109' group: Operation not permitted

Without `capability setuid`, the qemu guest log file contained:
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.11 -enable-kvm -m 512 -smp 1 -name dm1 -uuid 79d03a71-3be6-19df-1070-791239480888 -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/dm1.monitor,server,nowait -monitor chardev:monitor -boot c -drive file=/var/vm/dm1/disk0.qcow2,if=ide,index=0,boot=on -drive file=/var/vm/dm1/disk1.qcow2,if=ide,index=1 -net nic,macaddr=52:54:00:bf:75:90,vlan=0,model=virtio,name=virtio.0 -net tap,fd=50,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus
libvir: QEMU error : cannot change to '104' user: Operation not permitted

I don't really know if these changes were the right thing to do, but it did allow me to boot the VMs with virsh.

jad@kvmhost:~$ lsb_release -rd
Description: Ubuntu 10.04 LTS
Release: 10.04

jad@kvmhost:~$ apt-cache policy libvirt-bin kvm qemu-kvm
libvirt-bin:
  Installed: 0.7.5-5ubuntu27
  Candidate: 0.7.5-5ubuntu27
  Version table:
 *** 0.7.5-5ubuntu27 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status
kvm:
  Installed: 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9
  Candidate: 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9
  Version table:
 *** 1:84+dfsg-0ubuntu16+0.12.3+noroms+0ubuntu9 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status
qemu-kvm:
  Installed: 0.12.3+noroms-0ubuntu9
  Candidate: 0.12.3+noroms-0ubuntu9
  Version table:
 *** 0.12.3+noroms-0ubuntu9 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

[Mathias Gug (mathiaz)]

Could attach the guest description (virsh dump-xml) to the bug? Could you also specify the complete command line used to connect to libvirtd with virsh?

[Thomas Mayer (thomas303)]

Hello,

I just updated from jaunty to karmic and then to lucid.
After that, I had the same problem and I could not boot my images any more. Maybe apparmor was installed automatically and caused the problem when starting a guest:

error: Failed to start domain 220_trxerdpd330_installtest
error: internal error unable to start guest: libvir: QEMU error : cannot change to '114' group: Operation not permitted

after adding the following lines in /etc/apparmor.d/libvirt-qemu the problem was solved:
capability setgid,
capability setuid,

The xml of my guest looks like this

<domain type='kvm'>
  <name>220_trxerdpd330_installtest</name>
  <uuid>87cb0f4f-1d8f-4e8b-2a1f-4cda94aca1ec</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <source file='/home/vms/220_trxerdpd330_installtest.img'/>
      <target dev='hda' bus='ide'/>
    </disk>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <interface type='bridge'>
      <mac address='00:16:36:4e:bd:fb'/>
      <source bridge='br0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target port='0'/>
    </serial>
    <console type='pty' tty='/dev/pts/4'>
      <source path='/dev/pts/4'/>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='6220' autoport='no' keymap='de'/>
  </devices>
</domain>

When I mount a readonly cd image, I get similar errors:

I think there are two issues:
1. libvirt should not chown/chgrp/chmod images, especially not readonly images
2. apparmor profile should correspond to libvirt.

[s450r1 (s450r1)]

Mathias,

The complete command line to start virsh was `sudo virsh`.

Here's the guest description for one of the guests:

jad@kvmhost:~$ sudo virsh dumpxml couchdb1
<domain type='kvm' id='1'>
  <name>couchdb1</name>
  <uuid>57861152-9d28-c67d-87c6-a0295a418121</uuid>
  <memory>2097152</memory>
  <currentMemory>2097152</currentMemory>
  <vcpu>3</vcpu>
  <os>
    <type arch='x86_64' machine='pc-0.11'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <source file='/var/vm/couchdb1/disk0.qcow2'/>
      <target dev='hda' bus='ide'/>
    </disk>
    <disk type='file' device='disk'>
      <source file='/var/vm/couchdb1/disk1.qcow2'/>
      <target dev='hdb' bus='ide'/>
    </disk>
    <interface type='bridge'>
      <mac address='52:54:00:c8:8c:c5'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
    </video>
  </devices>
  <seclabel type='dynamic' model='apparmor'>
    <label>libvirt-57861152-9d28-c67d-87c6-a0295a418121</label>
    <imagelabel>libvirt-57861152-9d28-c67d-87c6-a0295a418121</imagelabel>
  </seclabel>
</domain>

[Jamie Strandboge (jdstrand)]

s450r1, can you attach your /etc/libvirt/qemu.conf file?

[s450r1 (s450r1)]

Jamie,

Attached, but it appears to be all comments.

Thanks,
Jeff

On Thu, May 27, 2010 at 9:41 AM, Jamie Strandboge <email address hidden> wrote:
> s450r1, can you attach your /etc/libvirt/qemu.conf file?
>
> --
> setgid, setuid needed by /etc/apparmor.d/abstractions/libvirt-qemu
> https://bugs.launchpad.net/bugs/579584
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “libvirt” package in Ubuntu: Incomplete
>
> Bug description:
> I couldn't boot any guest VMs with virsh until I modified /etc/apparmor.d/abstractions/libvirt-qemu:
> jad@kvmhost:~$ sudo bzr diff /etc/apparmor.d/
> === modified file 'apparmor.d/abstractions/libvirt-qemu'
> --- apparmor.d/abstractions/libvirt-qemu        2010-04-30 15:33:20 +0000
> +++ apparmor.d/abstractions/libvirt-qemu        2010-05-12 17:26:56 +0000
> @@ -8,6 +8,8 @@
>   capability dac_override,
>   capability dac_read_search,
>   capability chown,
> +  capability setgid,
> +  capability setuid,
>
>   # this is needed with libcap-ng support, however it breaks a lot of things
>   # atm, so just silence the denial until libcap-ng works right. LP: #522845
>
> ... and restarted apparmor and libvirtd.
>
> Without `capability setgid`, the qemu guest log file contained:
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_
> AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.11 -enable-kvm -m 512 -smp 1 -name dm1 -u
> uid 79d03a71-3be6-19df-1070-791239480888 -chardev socket,id=monitor,path=/var/li
> b/libvirt/qemu/dm1.monitor,server,nowait -monitor chardev:monitor -boot c -drive
>  file=/var/vm/dm1/disk0.qcow2,if=ide,index=0,boot=on -drive file=/var/vm/dm1/disk1.qcow2,if=ide,index=1 -net nic,macaddr=52:54:00:bf:75:90,vlan=0,model=virtio,name=virtio.0 -net tap,fd=50,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus
> libvir: QEMU error : cannot change to '109' group: Operation not permitted
>
> Without `capability setuid`, the qemu guest log file contained:
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.11 -enable-kvm -m 512 -smp 1 -name dm1 -uuid 79d03a71-3be6-19df-1070-791239480888 -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/dm1.monitor,server,nowait -monitor chardev:monitor -boot c -drive file=/var/vm/dm1/disk0.qcow2,if=ide,index=0,boot=on -drive file=/var/vm/dm1/disk1.qcow2,if=ide,index=1 -net nic,macaddr=52:54:00:bf:75:90,vlan=0,model=virtio,name=virtio.0 -net tap,fd=50,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus
> libvir: QEMU error : cannot change to '104' user: Operation not permitted
>
> I don't really know if these changes were the right thing to do, but it did allow me to boot the VMs with virsh.
>
> jad@kvmhost:~$ lsb_release -rd
> Description:    Ubuntu 10.04 LTS
> Release:        10.04
>
> jad@kvmhost:~$ apt-cache policy libvirt-bin kvm qemu-kvm
> libvirt-bin:
>  Installed: 0.7.5-5ubuntu27
>  Candidate: 0.7.5-5ubuntu27
>  Version table:
>  *** 0.7.5-5ubuntu27 0
>        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
>        100 /var/lib/dpkg/status
> kvm:
>  Installed: 1:84+d...

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 0.7.5-5ubuntu29

---------------
libvirt (0.7.5-5ubuntu29) maverick; urgency=low

  * debian/apparmor/libvirt-qemu: allow setgid and setuid so qemu can drop
    privileges (LP: #579584)
 -- Jamie Strandboge <email address hidden> Thu, 10 Jun 2010 13:09:44 -0500

[jdobry (jdobry)]

Fixed for Maverick, but not for Lucid. It is also touched.

[jdobry (jdobry)]

not fixed on Lucid

[Jamie Strandboge (jdstrand)]

jdobry, please don't change the bug status for bugs that are fixed in the devel release but not fixed in earlier releases. Instead, nominate this bug to be fixed in an earlier release.

[jdobry (jdobry)]

But that this bug is reported for Lucid and fixed for Maverick

[Jamie Strandboge (jdstrand)]

This recently came up on the libvirt mailing list:
https://www.redhat.com/archives/libvir-list/2010-September/msg00406.html

It appears that libvirt is aa_change_profile()ing before the DAC security driver can do its business. It seems that the ordering of the stacked security driver is wrong and that DAC driver should (always) go first, then the MAC (eg AppApparmor/SELinux) should come after. Before we push something to Lucid, I'd like to see upstream consensus on the fix (especially since we may want to change Maverick).

[Bryan McLellan (btm)]

I don't think this bug hits me on Lucid until I give libvirt a different group for the sock files. It'd be interesting if others seeing this bug are changing this value as well.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Serge Hallyn (serge-hallyn)]

I can't get libvirt in lucid to actually work with a non-root userid in /etc/libvirt/qemu.conf for launching vms. Therefore I don't believe this bug is valid there.