apparmor blocks VM installation when automatic UEFI firmware is set

[Katerina Koukiou]

# lsb_release -rd
Description: Ubuntu 21.10
Release: 21.10

Package: apparmor
Version: 3.0.3-0ubuntu1

Package: virtinst
Version: 1:3.2.0-3

When trying to re-install an existing VM with uefi boot set up using the
recently introduced `--reinstall` option apparmor makes the installation
fail with the following error:

Could not open '/var/lib/libvirt/qemu/nvram/test_VARS.fd': Permission denied

Steps to reproduce:

Create a VM:

root@ubuntu:~# virt-install --connect qemu:///system --quiet --os-variant
fedora28 --memory 1024 --name test --wait -1 --disk size=1,format=qcow2
--print-xml 1 > /tmp/test1.xml

Edit the VM configuration to enable automatic UEFI boot by changing the
<os> like follows:

- <os>

+ <os firmware='efi'>

Define the VM:

root@ubuntu:~# virsh define /tmp/test1.xml

Start VM installation:

root@ubuntu:~# virt-install --connect qemu:///system --reinstall test --wait -1 --noautoconsole --cdrom /var/lib/libvirt/novell.iso --autostart
WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.

Starting install...
ERROR internal error: process exited while connecting to monitor: 2022-02-23T18:56:54.738510Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/test_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/var/lib/libvirt/qemu/nvram/test_VARS.fd': Permission denied
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start test
otherwise, please restart your installation.

Expected behavior:

VM installation will start without apparmor error.

Actual behavior:

The above denials happen:

audit: type=1400 audit(1645796875.169:132): apparmor="DENIED" operation="open" profile="libvirt-68567d5b-c2c1-4256-9931-ce675df2f9b0" name="/var/lib/libvirt/qemu/nvram/test_VARS.fd" pid=4909 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055

same thing later on for "k" (locking)

audit: type=1400 audit(1645796969.776:151): apparmor="DENIED" operation="file_lock" profile="libvirt-68567d5b-c2c1-4256-9931-ce675df2f9b0" name="/usr/share/OVMF/OVMF_CODE_4M.secboot.fd" pid=5125 comm="qemu-system-x86" requested_mask="k" denied_mask="k" fsuid=64055 ouid=0

and stop the installation.

[Martin Pitt (pitti)]

/etc/apparmor.d/abstractions/libvirt-qemu is shipped by libvirt-daemon-system, reassigning. I can reproduce this, and I'll attempt to work on a fix. I'll update the Debian bug as well.

Complete copy&paste-able reproducer:

virt-install --connect qemu:///system --quiet --os-variant fedora28 --memory 128 --name test --wait -1 --disk size=0.125,format=qcow2 --graphics vnc,listen=127.0.0.1 --graphics spice,listen=127.0.0.1 --print-xml 1 | sed "s/<os/& firmware='efi'/" > /tmp/test1.xml
virsh define /tmp/test1.xml
touch /var/lib/libvirt/novell.iso
virt-install --connect qemu:///system --reinstall test --wait -1 --noautoconsole --cdrom /var/lib/libvirt/novell.iso --autostart

[Martin Pitt (pitti)]

I came up with this patch:

--- /etc/apparmor.d/abstractions/libvirt-qemu.orig 2022-01-22 18:22:57.000000000 +0000
+++ /etc/apparmor.d/abstractions/libvirt-qemu 2022-02-25 13:54:22.075405809 +0000
@@ -85,7 +85,7 @@
   /usr/share/misc/sgabios.bin r,
   /usr/share/openbios/** r,
   /usr/share/openhackware/** r,
- /usr/share/OVMF/** r,
+ /usr/share/OVMF/** rk,
   /usr/share/ovmf/** r,
   /usr/share/proll/** r,
   /usr/share/qemu-efi/** r,
@@ -249,5 +249,8 @@
   / r, # harmless on any lsb compliant system
   /sys/bus/nd/devices/{,**/} r,

+ # required for QEMU accessing UEFI nvram variables
+ /**/nvram/*_VARS.fd rwk,
+
   # Site-specific additions and overrides. See local/README for details.
   #include <local/abstractions/libvirt-qemu>

After

   systemctl reload apparmor.service; systemctl restart libvirtd

the reproducer works fine.

I'll send it to libvirt upstream now.

[Martin Pitt (pitti)]

I sent the proposed and tested fix upstream: https://gitlab.com/libvirt/libvirt/-/merge_requests/140

[Christian Ehrhardt  (paelzer)]

Thank you Martin and Katerina, for discussing it here and for driving it upstream.
I'm on a sprint this week and I know Martin "could" do all of this on his own. For now I tagged it so I can revisit the case more easily, if you need our help later to land it in Jammy let us know.

[Christian Ehrhardt  (paelzer)]

I finally had more than 20 seconds to look at this, commented in the upstream PR with a line that would be great to be added to match one common case that I think isn't covered yet.

[Martin Pitt (pitti)]

Thanks Christian. I updated the upstream PR. I just don't want to apply a patch just to Ubuntu. Once it lands upstream, I backport it, send it to Debian, and *then* I'm happy to apply it to Jammy -- there should still be enough time before the freeze, right? (Would be nice to have that in the LTS, to avoid regressions with cockpit-machines)

[Christian Ehrhardt  (paelzer)]

Yes that approach should be fine and I want it in Jammy as well.
We no more sync from Debian, so as soon as it is upstream committed we can consider adding it here.

[Christian Ehrhardt  (paelzer)]

I just approved it upstream, if no one else merges it let me know and I can give it another look.

[Christian Ehrhardt  (paelzer)]

I did the upstream merge, would you prepare a libvirt upload or should I?

[Martin Pitt (pitti)]

Fix landed upstream: https://gitlab.com/libvirt/libvirt/-/commit/7aec69b7fb9d0cfe8b7203473764c205b28d2905

[Martin Pitt (pitti)]

I sent https://salsa.debian.org/libvirt-team/libvirt/-/merge_requests/135 to update Debian. Unfortunately that does not build right now due to the inconsistent state of the packaging git. But the patch itself backports fairly cleanly.

I'll upload to Jammy next.

[Martin Pitt (pitti)]

I did a test build in my PPA: https://launchpad.net/~pitti/+archive/ubuntu/fixes

I re-ran the reproducer on current Jammy to confirm the bug, then updated to the PPA, and re-ran the last virt-install command. That succeeded.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 8.0.0-1ubuntu5

---------------
libvirt (8.0.0-1ubuntu5) jammy; urgency=medium

  * apparmor: Fix QEMU access for UEFI variable files. Backported from
    upstream master commit 7aec69b7fb9d0c. (Closes: #1006324, LP: #1962035)
    Refresh apparmor_profiles_local_include.patch to resolve the conflict.

 -- Martin Pitt <email address hidden> Wed, 09 Mar 2022 13:43:40 +0100

[Christian Ehrhardt  (paelzer)]

Migrated and I've not yet seen the world explode :-)
Thank you all!

[Alan Baghumian (alanbach)]

I just came across this bug on Focal/ARM64.

Applying the patch on comment #2 resolved the issue.

Would you be able to back-port this fix to Focal?

ii libvirt-clients 6.0.0-0ubuntu8.16 arm64 Programs for the libvirt library
ii libvirt-daemon 6.0.0-0ubuntu8.16 arm64 Virtualization daemon
ii libvirt-daemon-driver-qemu 6.0.0-0ubuntu8.16 arm64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.16 arm64 Virtualization daemon RBD storage driver
ii libvirt-daemon-system 6.0.0-0ubuntu8.16 arm64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 6.0.0-0ubuntu8.16 arm64 Libvirt daemon configuration files (systemd)
ii libvirt0:arm64 6.0.0-0ubuntu8.16 arm64 library for interfacing with different virtualization systems