[Impact]
* Attaching of virtual functions (SR-IOV) not possible without manually
modification of apparmor rules
* libvirt/qemu guest logfiles can not state the program that sent the
shutdown signal
* on ppc64el when starting on SMT enabled it failes (expected) but does
not present the hint about SMT being the reasons that was added in
>=zesty
* various apparmor denials in the log hide other issues
* Fix by opening up apparmor rules as needed for these use-cases
[Test Case]
* The rules covere a set of use cases, outlining them all here. Some only
apply to >=Zesty, so I mark each subtest with release names.
* All start with "Spawn a kvm guest e.g. via uvtool-libvirt"
* A) >=Xenial - shutdown message
- A1 - start the guest
- A2 - stop the guest
- A3 - check the log in /var/log/libvirt/qemu/<guestname>.log
- A-check: without the fix the log ends with <unknown process>
With the fix it shall contain the name of the process
stopping it instead of. So libvirt shutdown will show
libvirtd, kill from bash shows bash, ...)
* B) >=Xenial
- B1 - prepare a hotplug xml and a SR-IOV device - that depends on your
system, one example is outlined here:
# Prep VFs and attach
$ echo 4 | sudo tee /sys/bus/pci/devices/0005\:01\:00.0/sriov_numvfs
$ sudo modprobe vfio-pci
$ lspci -n -s 0005:01:01.3
0005:01:01.3 0200: 10df:e228 (rev 10)
$ cat VF-5.1.1.3.xml
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0005' bus='0x01' slot='0x01'
function='0x3'/>
</source>
</hostdev>
- B2 until bug 1679704 is fixed you need to manually increase the
locked memory limit of the qemu process before going on
- B3 attach the device
$ virsh attach-device z-test VF-5.1.1.3.xml
- B-check: without the fix it will fail for a set of apparmor denials
with the fix the attaching will succeed
* C) >=Zesty
- C1 - on a ppc64el system start and stop a guest
- C-check: without the fix apparmor will hold many denies for ppc64_cpu
and/or accessing /sys/devices/system/cpu/...
With the fix not only the denies are gone, but also if you
run on a system with SMT enabled (kvm will fail there) it
will report to you in the log about "Error: You must disable
SMT ..."
[Regression Potential]
* We are only further "opening up" the current apparmor profiles. So we
should not end up affecting existing use cases by new blocks.
* There is a very slight chance that due to accesses being allowed follow
on actions are triggered which people are not used to like "the VF hot
attach actually working". So if one relied on it not working it will be
different for them.
* Further I made some brainstorming on the potential effects but the only
other one that came to my mind being a regression of some sort is that
it now correctly states what send the shutdown signal. So when qemu did
not end by itself (guest stops) but is stopped (shutdown) the old
message was:
terminating on signal 15 from pid 22938 (<unknown process>)
And now will be like:
terminating on signal 15 from pid 7714 (/usr/sbin/libvirtd)
For the hopefully super unlikely case that someone e.g. grepped for
unknown here it might be a regression.
And making that message more useful (along with getting rid of the
associated aa deny is part of the intended fix).
[Other Info]
* Tested manually and via regression tests in advance on bileto tickets.
So all are green there now, there were two unrelated in some dep8 tests
due to what seems transient network timeouts (worked on retry) - if
showing up in proposed as well we will likely have to retrigger the
tests until good. Latest ticket for these to look at sniff builds:
=> https://bileto.ubuntu.com/#/ticket/2738
* Upload will be directly, since in some of the releases I will need
dpdk-genversion -v which bileto does not support (and it doesn't
support publishing selective releases) yet I wanted to show the extra
tests and checks made and therefore using bileto was beneficial even if
eventually uploading "old style"
-----
By debugging various bugs I've found several entries missing in the libvirt-qemu apparmor abstraction.
Those issues - now that they are understood - are taken out of the original bugs to focus on the remaining debugging there.
None of these are per Guest, so add to /etc/apparmor.d/abstractions/libvirt-qemu
The missing entries as identified are:
#1
For virtual functions the generic vfio interface
/etc/apparmor.d/abstractions/libvirt-qemu like:
# allow guest access to the generic base vfio interface (LP: #1678322)
/dev/vfio/vfio rw,
I checked with the security team and also according to https://www.kernel.org/doc/Documentation/vfio.txt this base interface should be save to be allowed.
#2
Binaries for ppc64el based checks in the kvm wrapper script.
That wrapper extension about smt awareness was tested and works fine as-is, but not when driven via libvirt due to the profile restrictions.
apparmor="DENIED" operation="exec" profile="libvirt-88b15add-b290-431d-9e49-fa771588f2f5" name="/usr/sbin/ppc64_cpu" pid=9539 comm="kvm" requested_mask="x" denied_mask="x" fsuid=64055 ouid=0
apparmor="DENIED" operation="exec" profile="libvirt-88b15add-b290-431d-9e49-fa771588f2f5" name="/bin/grep" pid=9541 comm="kvm" requested_mask="x" denied_mask="x" fsuid=64055 ouid=0
#3
Qemu tries to read who killed it to report on shutdown
[ 518.615420] audit: type=1400 audit(1491467132.255:21): apparmor="DENIED" operation="open" profile="libvirt-88b15add-b290-431d-9e49-fa771588f2f5" name="/proc/7989/cmdline" pid=9531 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=64055 ouid=0
The following should be rather save
/proc/*/cmdline r,
That will make the <unkown process> in logs like the following more readable:
"qemu-system-ppc64: terminating on signal 15 from pid 10924 (<unknown process>)"
Prepared a Test PPA at
https:/
Once tested on the issues and regressions we should shove that into zesty asap to meet the Freeze deadlines (or make it a very early update).