USB hot attach does not work (hostdev functionality only partially implemented)

[Serge Hallyn]

1. fired up a pre-existing vm

 virsh start mav-serv1

2. plugged in a usb drive

3. found the usb address using lsusb, which gave me

Bus 002 Device 006: ID 1058:1023 Western Digital Technologies, Inc.

4. defined a xml file with the device info:
<hostdev mode='subsystem' type='usb'>
 <source>
  <vendor id='0x1058'/>
  <product id='0x1023'/>
 </source>
</hostdev>

5. passed the usb drive to the vm

 sudo virsh attach-device mav-serv1 /tmp/a.xml

Expected results:

 lsusb in the VM shows the device

Actual results:

 lsusb in the VM does not show the device until I stop apparmor using
  sudo /etc/init.d/apparmor stop
  sudo /etc/init.d/apparmor teardown
 As soon as I do that, it shows up in the guest.

[Serge Hallyn (serge-hallyn)]

I see a lot of messages in syslog looking like:

Sep 16 19:14:46 sergelap kernel: [41633.228810] type=1400 audit(1284682486.737:309): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/
usb/002/009" pid=20410 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117

I'll append the profile contents after I re-rerun the test.

[Serge Hallyn (serge-hallyn)]

Actually, the first time that i run virsh atach-device, i get

root@sergelap:~# sudo virsh attach-device maverick2 /tmp/a.xml
error: Failed to attach device from /tmp/a.xml
error: operation failed: adding usb-host,hostbus=002,hostaddr=016,id=hostdev0 device failed: husb: using sys file-system with /dev/bus/usb

Syslog then claims:

Sep 16 21:20:20 sergelap libvirtd: 21:20:20.475: warning : qemudParsePCIDeviceStrs:1422 : Unexpected exit status '1', qemu probably failed
Sep 16 21:20:20 sergelap libvirtd: 21:20:20.479: error : qemuMonitorTextAddDevice:2340 : operation failed: adding usb-host,hostbus=002,hostaddr=016,id=hostdev0 device failed: husb: using sys file-system with /dev/bus/usb#015#012
Sep 16 21:20:20 sergelap kernel: [49160.665916] type=1400 audit(1284690020.467:344): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117

The second time it claims succes:

root@sergelap:~# virsh attach-device maverick2 /tmp/a.xml
Device attached successfully

but starts filling the logs with the error message:

Sep 16 21:21:35 sergelap libvirtd: 21:21:35.745: warning : qemudParsePCIDeviceStrs:1422 : Unexpected exit status '1', qemu probably failed
Sep 16 21:21:36 sergelap kernel: [49236.685478] type=1400 audit(1284690096.547:345): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:38 sergelap kernel: [49238.685623] type=1400 audit(1284690098.547:346): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:40 sergelap kernel: [49240.686528] type=1400 audit(1284690100.547:347): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:42 sergelap kernel: [49242.685611] type=1400 audit(1284690102.557:348): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:44 sergelap kernel: [49244.686534] type=1400 audit(1284690104.557:349): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:46 sergelap kernel: [49246.687311] type=1400 audit(1284690106.567:350): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:48 sergelap kernel: [49248.688348] type=1400 audit(1284690108.567:351): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5...

[Serge Hallyn (serge-hallyn)]

The profile just reads:

root@sergelap:~# cat /etc/apparmor.d/libvirt/libvirt-27f9f30c-5632-2504-5006-e329fcf24d32
#
# This profile is for the domain whose UUID matches this file.
#

#include <tunables/global>

profile libvirt-27f9f30c-5632-2504-5006-e329fcf24d32 {
  #include <abstractions/libvirt-qemu>
  #include <libvirt/libvirt-27f9f30c-5632-2504-5006-e329fcf24d32.files>

}

[Jamie Strandboge (jdstrand)]

Does it work properly if you disable the apparmor driver? This can be done by adjusting /etc/libvirt/qemu.conf to have:
security_driver = "none"

Then restarting libvirt with:
$ sudo stop libvirt-bin
$ sudo start libvirt-bin

Then verifying with 'virsh capabilities' that the following is not present in the host section:
    <secmodel>
      <model>apparmor</model>
      <doi>0</doi>
    </secmodel>

[Jamie Strandboge (jdstrand)]

Actually, I see this here too.

[Jamie Strandboge (jdstrand)]

In investigating this, libvirt is not calling virt-aa-helper at all. Now to find out why.

[Jamie Strandboge (jdstrand)]

I remember the issue now. The problem is that AppArmorSetSecurityHostdevLabel() is only a stub. It is a stub because while we would want to do something like this:
    if (profile_loaded(secdef->imagelabel) >= 0) {
        if (load_profile(drv, secdef->imagelabel, vm, NULL) < 0) {
            virSecurityReportError(VIR_ERR_INTERNAL_ERROR,
                                   _("cannot update AppArmor profile "
                                     "\'%s\'"),
                                   secdef->imagelabel);
            return -1;
        }
    }

we cannot because the 'vm' xml that is passed to virt-aa-helper in this implementation does not have the new xml for the newly attached drive. Indeed

This is not a regression over lucid since it has the same problem and karmic didn't do it either. However, in karmic the /etc/apparmor.d/abstractions/libvirt-qemu had:
  # WARNING: uncommenting these gives the guest direct access to host hardware.
  # This is required for USB pass through but is a security risk. You have been
  # warned.
  #/sys/bus/usb/devices/ r,
  #/sys/devices/*/*/usb[0-9]*/** r,
  #/dev/bus/usb/*/[0-9]* rw,

But in lucid and later we have:
  # For hostdev access. The actual devices will be added dynamically
  /sys/bus/usb/devices/ r,
  /sys/devices/*/*/usb[0-9]*/** r,

part of the functionality for hostdev access was added to virt-aa-helper in lucid and later, but clearly not enough to fully fix hostdev hot attach. So the workaround is to update /etc/apparmor.d/libvirt-qemu to have:
  /dev/bus/usb/*/[0-9]* rw,

That does give all guests access to any usb hardware of course, which is not all that great. People can add a more specific rule to /etc/apparmor.d/libvirt/libvirt-<uuid> (not the .files file!) to limit access for a specific guest to a specific USB device.

[Jamie Strandboge (jdstrand)]

Attached is a patch to fix this for maverick. I am submitting upstream and want to get their feedback before trying to get into maverick. As such, this is likely SRU material. It is something that can be considered for Lucid once we have upstream approval.

[Jamie Strandboge (jdstrand)]

This has been fixed in Natty.

[Rolf Leggewie (r0lf)]

maverick has seen the end of its life and is no longer receiving any updates. Marking the maverick task for this ticket as "Won't Fix".

[Rolf Leggewie (r0lf)]

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".