Libvirt 8.0.0 Error When Starting Windows 11 VM "'/var/lib/libvirt/qemu/nvram/win11_VARS.fd': Permission denied"

[Jesse Lopez]

- Windows 11 guest OS
- swtpm TPM emulator version 0.6.1
- TPM with Model "TIS" and version 2.0
- Libvirt 8.0.0

I am unable to start a Windows 11 VM with libvirt/QEMU and see the following error message. This happens after I add firmware="efi" in the VM's XML configuration and attempt to start the VM. This is caused by AppArmor.

<os firmware="efi">
    <type arch="x86_64" machine="pc-q35-6.1">hvm</type>
    <boot dev="hd"/>
</os>

Here is the error:

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1329, in startup
    self._backend.create()
  File "/usr/local/lib/python3.8/dist-packages/libvirt.py", line 1353, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: process exited while connecting to monitor: 2023-07-12T23:10:04.929455Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/win11_x64_1_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/var/lib/libvirt/qemu/nvram/win11_x64_1_VARS.fd': Permission denied

Here are the AppArmor logs for qemu:

apparmor="DENIED" operation="open" profile="libvirt-8ac25d83-8270-4f50-a201-18264ff41652" name="/var/lib/libvirt/qemu/nvram/win11_VARS.fd" pid=27563 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

apparmor="DENIED" operation="file_lock" profile="libvirt-8ac25d83-8270-4f50-a201-18264ff41652" name="/var/lib/libvirt/qemu/nvram/win11_VARS.fd" pid=26033 comm="qemu-system-x86" requested_mask="k" denied_mask="k" fsuid=0 ouid=0

apparmor="DENIED" operation="file_lock" profile="libvirt-8ac25d83-8270-4f50-a201-18264ff41652" name="/usr/share/OVMF/OVMF_CODE_4M.secboot.fd" pid=24034 comm="qemu-system-x86" requested_mask="k" denied_mask="k" fsuid=0 ouid=0

apparmor="DENIED" operation="open" profile="libvirt-8ac25d83-8270-4f50-a201-18264ff41652" name="/sys/kernel/mm/transparent_hugepage/enabled" pid=23369 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

apparmor="DENIED" operation="mknod" profile="libvirt-8ac25d83-8270-4f50-a201-18264ff41652" name="/run/libvirt/qemu/swtpm/3-win11-swtpm.sock" pid=33103 comm="swtpm" requested_mask="c" denied_mask="c" fsuid=106 ouid=106

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

Found solution -

1. Add the following lines to the end of /etc/apparmor.d/abstractions/libvirt-qemu

/var/lib/libvirt/qemu/nvram/* rwk,
/sys/kernel/mm/transparent_hugepage/enabled r,
/usr/share/OVMF/OVMF_CODE_4M.secboot.fd rk,
/run/libvirt/qemu/swtpm/* rwk,

2. Reload apparmor, restart libvirtd:

sudo systemctl reload apparmor.service
sudo systemctl restart libvirtd