USB hot attach does not work (hostdev functionality only partially implemented)

Bug #640993 reported by Serge Hallyn
30
This bug affects 5 people
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Medium
Jamie Strandboge
Lucid
Undecided
Unassigned
Maverick
Medium
Unassigned
Natty
Medium
Jamie Strandboge

Bug Description

1. fired up a pre-existing vm

 virsh start mav-serv1

2. plugged in a usb drive

3. found the usb address using lsusb, which gave me

Bus 002 Device 006: ID 1058:1023 Western Digital Technologies, Inc.

4. defined a xml file with the device info:
<hostdev mode='subsystem' type='usb'>
 <source>
  <vendor id='0x1058'/>
  <product id='0x1023'/>
 </source>
</hostdev>

5. passed the usb drive to the vm

 sudo virsh attach-device mav-serv1 /tmp/a.xml

Expected results:

 lsusb in the VM shows the device

Actual results:

 lsusb in the VM does not show the device until I stop apparmor using
  sudo /etc/init.d/apparmor stop
  sudo /etc/init.d/apparmor teardown
 As soon as I do that, it shows up in the guest.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

I see a lot of messages in syslog looking like:

Sep 16 19:14:46 sergelap kernel: [41633.228810] type=1400 audit(1284682486.737:309): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/
usb/002/009" pid=20410 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117

I'll append the profile contents after I re-rerun the test.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :
Download full text (3.3 KiB)

Actually, the first time that i run virsh atach-device, i get

root@sergelap:~# sudo virsh attach-device maverick2 /tmp/a.xml
error: Failed to attach device from /tmp/a.xml
error: operation failed: adding usb-host,hostbus=002,hostaddr=016,id=hostdev0 device failed: husb: using sys file-system with /dev/bus/usb

Syslog then claims:

Sep 16 21:20:20 sergelap libvirtd: 21:20:20.475: warning : qemudParsePCIDeviceStrs:1422 : Unexpected exit status '1', qemu probably failed
Sep 16 21:20:20 sergelap libvirtd: 21:20:20.479: error : qemuMonitorTextAddDevice:2340 : operation failed: adding usb-host,hostbus=002,hostaddr=016,id=hostdev0 device failed: husb: using sys file-system with /dev/bus/usb#015#012
Sep 16 21:20:20 sergelap kernel: [49160.665916] type=1400 audit(1284690020.467:344): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117

The second time it claims succes:

root@sergelap:~# virsh attach-device maverick2 /tmp/a.xml
Device attached successfully

but starts filling the logs with the error message:

Sep 16 21:21:35 sergelap libvirtd: 21:21:35.745: warning : qemudParsePCIDeviceStrs:1422 : Unexpected exit status '1', qemu probably failed
Sep 16 21:21:36 sergelap kernel: [49236.685478] type=1400 audit(1284690096.547:345): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:38 sergelap kernel: [49238.685623] type=1400 audit(1284690098.547:346): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:40 sergelap kernel: [49240.686528] type=1400 audit(1284690100.547:347): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:42 sergelap kernel: [49242.685611] type=1400 audit(1284690102.557:348): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:44 sergelap kernel: [49244.686534] type=1400 audit(1284690104.557:349): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:46 sergelap kernel: [49246.687311] type=1400 audit(1284690106.567:350): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5632-2504-5006-e329fcf24d32" name="/dev/bus/usb/002/016" pid=10138 comm="kvm" requested_mask="rw" denied_mask="rw" fsuid=117 ouid=117
Sep 16 21:21:48 sergelap kernel: [49248.688348] type=1400 audit(1284690108.567:351): apparmor="DENIED" operation="open" parent=1 profile="libvirt-27f9f30c-5...

Read more...

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

The profile just reads:

root@sergelap:~# cat /etc/apparmor.d/libvirt/libvirt-27f9f30c-5632-2504-5006-e329fcf24d32
#
# This profile is for the domain whose UUID matches this file.
#

#include <tunables/global>

profile libvirt-27f9f30c-5632-2504-5006-e329fcf24d32 {
  #include <abstractions/libvirt-qemu>
  #include <libvirt/libvirt-27f9f30c-5632-2504-5006-e329fcf24d32.files>

}

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Does it work properly if you disable the apparmor driver? This can be done by adjusting /etc/libvirt/qemu.conf to have:
security_driver = "none"

Then restarting libvirt with:
$ sudo stop libvirt-bin
$ sudo start libvirt-bin

Then verifying with 'virsh capabilities' that the following is not present in the host section:
    <secmodel>
      <model>apparmor</model>
      <doi>0</doi>
    </secmodel>

Changed in libvirt (Ubuntu):
assignee: nobody → Jamie Strandboge (jdstrand)
status: New → Incomplete
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Actually, I see this here too.

Changed in libvirt (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

In investigating this, libvirt is not calling virt-aa-helper at all. Now to find out why.

Changed in libvirt (Ubuntu):
status: Confirmed → In Progress
importance: Undecided → Medium
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

I remember the issue now. The problem is that AppArmorSetSecurityHostdevLabel() is only a stub. It is a stub because while we would want to do something like this:
    if (profile_loaded(secdef->imagelabel) >= 0) {
        if (load_profile(drv, secdef->imagelabel, vm, NULL) < 0) {
            virSecurityReportError(VIR_ERR_INTERNAL_ERROR,
                                   _("cannot update AppArmor profile "
                                     "\'%s\'"),
                                   secdef->imagelabel);
            return -1;
        }
    }

we cannot because the 'vm' xml that is passed to virt-aa-helper in this implementation does not have the new xml for the newly attached drive. Indeed

This is not a regression over lucid since it has the same problem and karmic didn't do it either. However, in karmic the /etc/apparmor.d/abstractions/libvirt-qemu had:
  # WARNING: uncommenting these gives the guest direct access to host hardware.
  # This is required for USB pass through but is a security risk. You have been
  # warned.
  #/sys/bus/usb/devices/ r,
  #/sys/devices/*/*/usb[0-9]*/** r,
  #/dev/bus/usb/*/[0-9]* rw,

But in lucid and later we have:
  # For hostdev access. The actual devices will be added dynamically
  /sys/bus/usb/devices/ r,
  /sys/devices/*/*/usb[0-9]*/** r,

part of the functionality for hostdev access was added to virt-aa-helper in lucid and later, but clearly not enough to fully fix hostdev hot attach. So the workaround is to update /etc/apparmor.d/libvirt-qemu to have:
  /dev/bus/usb/*/[0-9]* rw,

That does give all guests access to any usb hardware of course, which is not all that great. People can add a more specific rule to /etc/apparmor.d/libvirt/libvirt-<uuid> (not the .files file!) to limit access for a specific guest to a specific USB device.

summary: - passing a usb device in maverick with apparmor enabled fails
+ USB hot attach does not work (hostdev functionality only partially
+ implemented)
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Attached is a patch to fix this for maverick. I am submitting upstream and want to get their feedback before trying to get into maverick. As such, this is likely SRU material. It is something that can be considered for Lucid once we have upstream approval.

tags: added: apparmor
Changed in libvirt (Ubuntu):
milestone: none → maverick-updates
Changed in libvirt (Ubuntu Lucid):
status: New → Triaged
milestone: none → lucid-updates
tags: added: patch
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

This has been fixed in Natty.

Changed in libvirt (Ubuntu Maverick):
assignee: Jamie Strandboge (jdstrand) → nobody
milestone: maverick-updates → none
status: In Progress → Triaged
Changed in libvirt (Ubuntu Natty):
status: In Progress → Fix Released
milestone: maverick-updates → none
Revision history for this message
Rolf Leggewie (r0lf) wrote :

maverick has seen the end of its life and is no longer receiving any updates. Marking the maverick task for this ticket as "Won't Fix".

Changed in libvirt (Ubuntu Maverick):
status: Triaged → Won't Fix
Revision history for this message
Rolf Leggewie (r0lf) wrote :

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".

Changed in libvirt (Ubuntu Lucid):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers