ppc64el virsh start fails

[Scott Moser]

===========================================
Impact: cannot start libvirt-qemu VMs on ppc64el
Test case: See below in the description.
Regression potential: this debdiff only adds apparmor permissions which are already being granted in vivid, so no new regressions should be possible
===========================================
$ cat > foobar.xml <<EOF
<domain type="kvm">
  <name>foobar</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <os>
    <type arch="ppc64" machine="pseries">hvm</type>
  </os>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset="utc"/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64</emulator>
    <console type="pty"/>
  </devices>
</domain>
EOF

$ sudo virsh define foobar.xml
Domain foobar defined from foobar.xml

$ sudo virsh start foobar
error: Failed to start domain foobar
error: internal error: process exited while connecting to monitor: Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
Can't open directory /proc/device-tree/cpus/
qemu: hardware error: qemu: could not load LPAR rtas '/usr/share/qemu/spapr-rtas.bin'

CPU #0:
NIP 0000000000000000 LR 0000000000000000 CTR 0000000000000000 XER 0000000000000000
MSR 0000000000000000 HID0 0000000000000000 HF 0000000000000000 idx 0
TB 00000000 00000000 DECR 00000000
GPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR16 0000000000000000 00000000000000

ProblemType: Bug
DistroRelease: Ubuntu 14.10
Package: libvirt-bin 1.2.8-0ubuntu4
ProcVersionSignature: User Name 3.16.0-17.23-generic 3.16.3
Uname: Linux 3.16.0-17-generic ppc64le
ApportVersion: 2.14.7-0ubuntu2
Architecture: ppc64el
Date: Fri Sep 26 17:43:55 2014
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcLoadAvg: 0.08 0.03 0.05 1/1176 28307
ProcLocks:
 1: POSIX ADVISORY WRITE 84870 00:11:478688 0 0
 2: FLOCK ADVISORY WRITE 2515 00:11:45214 0 EOF
 3: POSIX ADVISORY WRITE 2369 00:11:50266 0 EOF
 4: POSIX ADVISORY WRITE 2567 00:11:47202 0 EOF
ProcSwaps:
 Filename Type Size Used Priority
 /swap.img file 8388544 0 -1
ProcVersion: Linux version 3.16.0-17-generic (buildd@fisher03) (gcc version 4.9.1 (User Name 4.9.1-15ubuntu1) ) #23-User Name SMP Fri Sep 19 16:54:14 UTC 2014
SourcePackage: libvirt
UpgradeStatus: No upgrade log present (probably fresh install)
cpu_cores: Number of cores present = 20
cpu_coreson: Number of cores online = 19
cpu_smt: SMT is off
modified.conffile..etc.libvirt.qemu.conf: [inaccessible: [Errno 13] Permission denied: '/etc/libvirt/qemu.conf']

[Serge Hallyn (serge-hallyn)]

Thanks for submitting this bug. This is odd given that the libirt-qemu
abstraction is already being given specifically access to anything
under /proc/device-tree. Could you please attach the contents
of /etc/apparmor.d/usr.sbin.libvirt and /etc/apparmor.d/abstractions/libvirt-qemu
on the host, as well as relevant DENIED messages from /var/log/syslog?

 status: incomplete
 importance: high

[Scott Moser (smoser)]

/etc/apparmor.d/usr.sbin.libvirt is the stock as installed via libvirt-bin at 1.2.8-0ubuntu4 .
I've made no changes to /etc/apparmor.d/abstractions/libvirt-qemu
running virsh start as show above does add the following to dmesg:

[394460.246874] audit_printk_skb: 6 callbacks suppressed
[394460.246878] audit: type=1400 audit(1412090246.041:126): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" pid=1838 comm="apparmor_parser"
[394460.247104] audit: type=1400 audit(1412090246.041:127): apparmor="STATUS" operation="profile_load" profile="unconfined" name="qemu_bridge_helper" pid=1838 comm="apparmor_parser"
[394460.298262] audit: type=1400 audit(1412090246.093:128): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298293] audit: type=1400 audit(1412090246.093:129): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298307] audit: type=1400 audit(1412090246.093:130): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298321] audit: type=1400 audit(1412090246.093:131): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298353] audit: type=1400 audit(1412090246.093:132): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298366] audit: type=1400 audit(1412090246.093:133): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298379] audit: type=1400 audit(1412090246.093:134): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[394460.298393] audit: type=1400 audit(1412090246.093:135): apparmor="DENIED" operation="open" profile="libvirt-de3582cd-f37d-484c-8dde-10727cad60c0" name="/sys/firmware/devicetree/base/cpus/" pid=1840 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0

[Serge Hallyn (serge-hallyn)]

Thanks. So we need to add

/sys/firmware/devicetree/** r

to the libvirt-qemu template.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.2.8-0ubuntu9

---------------
libvirt (1.2.8-0ubuntu9) utopic; urgency=medium

  * libvirt-qemu apparmor template: add /sys/firmware/devicetree/** r
    (LP: #1374554)
 -- Serge Hallyn <email address hidden> Wed, 01 Oct 2014 17:09:05 -0500

[Breno Leitão (breno-leitao)]

I understand that the ppc64le slof is not t /sys/firmware/devicetree, so , the problem is still reproducible with newer version as 1.2.8-0ubuntu11. So, I am reopening this bug as it is not fixed with the line described in comment #4.

I added the line for /usr/share/slof and it seems to really solve this problem.

This is a patch that addes this line
--- /etc/apparmor.d/abstractions/libvirt-qemu.orig 2014-11-04 11:07:30.633029803 -0500
+++ /etc/apparmor.d/abstractions/libvirt-qemu 2014-11-04 11:07:58.721030420 -0500
@@ -76,6 +76,7 @@
   /usr/share/vgabios/** r,
   /usr/share/seabios/** r,
   /usr/share/ovmf/** r,
+ /usr/share/slof/** r,

   # access PKI infrastructure
   /etc/pki/libvirt-vnc/** r,

[Serge Hallyn (serge-hallyn)]

More changes are apparently needed.

[Scott Moser (smoser)]

verified this is still broken.
[1355493.845370] audit: type=1400 audit(1415183670.500:26): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-184f2f3d-e71b-4cb3-a58c-9feab5c361b6" pid=94288 comm="apparmor_parser"
[1355493.845600] audit: type=1400 audit(1415183670.500:27): apparmor="STATUS" operation="profile_load" profile="unconfined" name="qemu_bridge_helper" pid=94288 comm="apparmor_parser"
[1355493.946737] audit: type=1400 audit(1415183670.604:28): apparmor="DENIED" operation="open" profile="libvirt-184f2f3d-e71b-4cb3-a58c-9feab5c361b6" name="/usr/share/slof/spapr-rtas.bin" pid=94290 comm="qemu-system-ppc" requested_mask="r" denied_mask="r" fsuid=107 ouid=0
[1355497.187373] audit: type=1400 audit(1415183673.844:29): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-184f2f3d-e71b-4cb3-a58c-9feab5c361b6" pid=94317 comm="apparmor_parser"

and verified that this fixes:
$ sudo diff -u /etc/apparmor.d/abstractions/libvirt-qemu.dist /etc/apparmor.d/abstractions/libvirt-qemu
--- /etc/apparmor.d/abstractions/libvirt-qemu.dist 2014-11-05 10:35:35.572911000 +0000
+++ /etc/apparmor.d/abstractions/libvirt-qemu 2014-11-05 10:36:05.392911000 +0000
@@ -76,6 +76,7 @@
   /usr/share/vgabios/** r,
   /usr/share/seabios/** r,
   /usr/share/ovmf/** r,
+ /usr/share/slof/** r,

   # access PKI infrastructure
   /etc/pki/libvirt-vnc/** r,

[Serge Hallyn (serge-hallyn)]

Thanks Scott and Breno, I'll push that fix.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.2.8-0ubuntu14

---------------
libvirt (1.2.8-0ubuntu14) vivid; urgency=medium

  [ Serge Hallyn ]
  * 9036-util-prepare-uri-for-libxml2-2.9.2.patch: fix FTBFS against new
    libxml 2.9.2 (LP: #1390637)

  [ Marc Deslauriers ]
  * SECURITY UPDATE: denial of service via virConnectListAllDomains
    - debian/patches/CVE-2014-3657.patch: fix domain deadlock in
      src/conf/domain_conf.c.
    - CVE-2014-3657
  * SECURITY UPDATE: xml information leak with read-only connections
    - debian/patches/CVE-2014-7823.patch: check for migratable flag in
      src/libvirt.c, src/remote/remote_protocol.x.
    - CVE-2014-7823
 -- Marc Deslauriers <email address hidden> Tue, 11 Nov 2014 13:14:00 -0500

[Chris J Arges (arges)]

Hello Scott, or anyone else affected,

Accepted libvirt into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libvirt/1.2.2-0ubuntu13.1.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris J Arges (arges)]

Hello Scott, or anyone else affected,

Accepted libvirt into utopic-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libvirt/1.2.8-0ubuntu11.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris J Arges (arges)]

I can verify this works on utopic.

2014-12-01 20:44:48.773+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-system-ppc64 -name foobar -S -machine pseries,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 5c9d2d1c-496c-4507-a1a6-3b70b2ce9d48 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/foobar.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -boot strict=on -device pci-ohci,id=usb,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 -msg timestamp=on
char device redirected to /dev/pts/3 (label charserial0)
root@modoc:~/vm# virsh list
 Id Name State
----------------------------------------------------
 4 foobar running

[Serge Hallyn (serge-hallyn)]

As discussed on irc, given that the fix consists of two added apparmor policy lines, and both are present (have just checked) and identical in the trusty and utopic proposed packages, and given that trusty support for ppc64el is more problematic, I am marking this verifcation-done based on the utopic confirmation.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.2.2-0ubuntu13.1.8

---------------
libvirt (1.2.2-0ubuntu13.1.8) trusty-proposed; urgency=medium

  * complete the 9p support: (LP: #1378434)
    - libvirt-qemu: add fowner and fsetid
    - virt-aa-helper: add 'l' to 9p file options
  * libvirt-qemu apparmor template: add /sys/firmware/devicetree/** r
    (LP: #1374554)
  * add mising apparmor permissions for slof (LP: #1374554)
 -- Serge Hallyn <email address hidden> Tue, 11 Nov 2014 16:39:22 -0600

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for libvirt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.2.8-0ubuntu11.2

---------------
libvirt (1.2.8-0ubuntu11.2) utopic-proposed; urgency=medium

  * complete the 9p support: (LP: #1378434)
    - libvirt-qemu: add fowner and fsetid
    - virt-aa-helper: add 'l' to 9p file options
  * add mising apparmor permissions for slof (LP: #1374554)
 -- Serge Hallyn <email address hidden> Tue, 11 Nov 2014 16:48:19 -0600