Libvirt snapshot doesn't update apparmor profile

[Dominique Ramaekers]

Libvirt doesn't update the VM's apparmor file when creating a external snapshot of a host with miltiple disks. It result in failing to create the snapshot.

There is also a Red Hat Bugzilla bug: https://bugzilla.redhat.com/show_bug.cgi?id=1746684#

root@CmsrvVH3:~# virsh -c qemu+ssh://root@CmsrvVH2/system snapshot-create-as CmsrvLAP2 --name onderhoudscript --disk-only --atomic --no-metadata --diskspec vda,file=/srv/poolVMS/CmsrvLAP2.onderhoudscript --diskspec vdb,file=/srv/poolVMS/CmsrvLAP2-SWAP.onderhoudscript --quiesce
fout: internal error: unable to execute QEMU command 'transaction': Could not create file: Permission denied

when I edit the apparmor profile of the guest and add this line:
"/srv/poolVMS/CmsrvLAP2*" rwk,
All works fine.

I can clearly see that libvirt doesn't add the line on the /etc/apparmor.d/libvirt/libvirt-[UUID].files

Also on guest with only one disk, everything works fine.

Some extra info:

root@CmsrvVH2:~# libvirtd --version
libvirtd (libvirt) 5.4.0

root@CmsrvVH2:~# cat /etc/*release*
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=19.10
DISTRIB_CODENAME=eoan
DISTRIB_DESCRIPTION="Ubuntu Eoan Ermine (development branch)"
...

[lars.dunemark (lars.dunemark-redhat-bugs)]

Created attachment 1609257
Domain file

Description of problem:
Creating a domain with multiple disks and trying to take an disk-only snapshot with external disk overlay fails with the error "Could not create file: Permission denied"

Version-Release number of selected component (if applicable):
Tested on 4.0.0, 5.0.0 and master (648c11c04cf1d45f37f4662ffb7952611ddb458c)

How reproducible:
Create a new domain for qemu with 2 disk connected. (dumpxml of my domain as attachemnt)

Steps to Reproduce:
1. snapshot-create-as --domain ubuntu18.04 --disk-only --atomic --diskspec vda,file=/var/lib/libvirt/images/ubuntu18.04-overlay.qcow2,snapshot=external --diskspec vdb,file=/var/lib/libvirt/images/ubuntu18.04-1-overlay.qcow2,snapshot=external

Actual results:
error: internal error: unable to execute QEMU command 'transaction': Could not create file: Permission denied

Expected results:
Domain snapshot 1567058757 created

Additional info:
When manually adding the path to vda overlay file in /etc/apparmor.d/libvirt/libvirt-a955728a-ac8f-4fcb-8bea-3e12fca826a7 as:
  "/var/lib/libvirt/images/ubuntu18.04-overlay.qcow2" rwk,

It works to take snapshot for both disk. So it looks like the apparmor is only updated with the last disk

[Christian Ehrhardt  (paelzer)]

Hi Dominique,
isn't that the same as I outlined on [1] ?

In case you think it is related could you for a check open /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper and remove the deny lines from /dev/sd to /dev/mapper; then reload via `sudo apparmor_parser -r /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper` and try again.

And if it is not the same, there I definitely did "snapshot of one disk with guests having multiple disks" there. So you might take a look what differs.
From a glimpse at your command I did none,none,none,snap,none while you try to snapshot two at once.

In case the above didn't help, could you attach a guest XML so that I can recreate a similar case?

[1]: https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1525310/comments/2

[Dominique Ramaekers (dominique-ramaekers)]

@paelzer,

I don't think my bug is a duplicate of 1525310...

With me, everything works fine from the start. Creating snapshots works fine also. Only with guest that has multiple disks, the snapshot fails.

Nonetheless, I tested the snapshot with the /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper lines removed (like you suggested) and the snapshot failed just the same.

I agree with Mr. Lars Dunemark who reported the bug on Bugzilla. Libvirt doesn't manage the /etc/apparmor.d/libvirt/libvirt-[UUID].files file correctly on snapshotting a guest with multiple disks.

[Christian Ehrhardt  (paelzer)]

Then please attach a guest XML of your case as there are too many ways to configure/attach disks to try all of them. I'll give it a retry myself then when I'm back to chime in with a +1 on the case then (if confirmed).

[Dominique Ramaekers (dominique-ramaekers)]

Please find xml attached.

[Christian Ehrhardt  (paelzer)]

From the description of Dominque this seemed a common case, so I tried with just qcow files and got it confirmed.

# Create basic guest (already has two disks)
uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=eoan
uvt-kvm create --password ubuntu eoan arch=amd64 release=eoan label=daily

# Add further disks for the test:
sudo qemu-img create -f qcow2 /var/lib/uvtool/libvirt/images/eoan-disk1.qcow 1G
sudo qemu-img create -f qcow2 /var/lib/uvtool/libvirt/images/eoan-disk2.qcow 1G
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/uvtool/libvirt/images/eoan-disk1.qcow'/>
      <target dev='vdc' bus='virtio'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/uvtool/libvirt/images/eoan-disk2.qcow'/>
      <target dev='vdd' bus='virtio'/>
    </disk>

The guest now looks like:
$ virsh domblklist eoan --details
 Type Device Target Source
--------------------------------------------------------------------------
 file disk vda /var/lib/uvtool/libvirt/images/eoan.qcow
 file disk vdb /var/lib/uvtool/libvirt/images/eoan-ds.qcow
 file disk vdc /var/lib/uvtool/libvirt/images/eoan-disk1.qcow
 file disk vdd /var/lib/uvtool/libvirt/images/eoan-disk2.qcow

Snapshot of single disk works:
$ virsh snapshot-create-as --domain eoan --disk-only --atomic --diskspec vda,snapshot=no --diskspec vdb,snapshot=no --diskspec vdc,file=/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow,snapshot=external --diskspec vdd,snapshot=no

The apparmor profile got the snapshot added as expected:
cat /etc/apparmor.d/libvirt/libvirt-72b929d2-389d-4c60-9f3b-4c3a8a98b4b0.files
...
  "/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow" rwk,

Snapshot of multiple disks fails:
virsh snapshot-create-as --domain eoan --disk-only --atomic --diskspec vda,snapshot=no --diskspec vdb,snapshot=no --diskspec vdc,file=/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow,snapshot=external --diskspec vdd,file=/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow,snapshot=external
error: internal error: unable to execute QEMU command 'transaction': Could not create file: Permission denied

None of the two paths got added to the apparmor profile.

Alongside that we see the expected apparmor denials.
 apparmor="DENIED" operation="open" profile="libvirt-72b929d2-389d-4c60-9f3b-4c3a8a98b4b0" name="/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow" pid=23603 comm="qemu-system-x86" requested_mask="wrc" denied_mask="wrc" fsuid=64055 ouid=64055

This proves the report.
I'll be out for a while after today, but I agree that we need to sort out what is missing in this case.
In the single snapshot case I've seen virt-aa-helper called to add a line, needs debugging where this fails with more than one snapshot target.

Until then one might as workaround try to snapshot each of the disks one by one (therefore only medium).

[Christian Ehrhardt  (paelzer)]

FYI: Also the dumpxml of the other reported case was at: https://bugzilla.redhat.com/attachment.cgi?id=1609257

[Christian Ehrhardt  (paelzer)]

logs:

2 disks:
Oct 15 13:14:07 e libvirtd[612]: internal error: unable to execute QEMU command 'transaction': Could not create file: Permission denied
Oct 15 13:14:08 e libvirtd[612]: Path '/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow' is not accessible: No such file or directory
Oct 15 13:14:08 e libvirtd[612]: Unable to tear down cgroup access on /var/lib/libvirt/images/eoan-disk1.snapshot1.qcow
Oct 15 13:14:08 e libvirtd[612]: internal error: child reported (status=125): unable to set user and group to '0:0' on '/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow': No such file or directory
Oct 15 13:14:08 e libvirtd[612]: Unable to restore security label on /var/lib/libvirt/images/eoan-disk1.snapshot1.qcow
Oct 15 13:14:08 e libvirtd[612]: Path '/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow' is not accessible: No such file or directory
Oct 15 13:14:08 e libvirtd[612]: Unable to tear down cgroup access on /var/lib/libvirt/images/eoan-disk2.snapshot1.qcow
Oct 15 13:14:08 e libvirtd[612]: internal error: child reported (status=125): unable to set user and group to '0:0' on '/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow': No such file or directory
Oct 15 13:14:08 e libvirtd[612]: Unable to restore security label on /var/lib/libvirt/images/eoan-disk2.snapshot1.qcow

One disk:
- no message
- new rule
  "/var/lib/libvirt/images/eoan-disk1.snapshot1.qcow" rwk,

This looks like a late call from labelling (comes with rwk by default).

Check what we get with one disk and two disks in GDB:
Num Type Disp Enb Address What
3 breakpoint keep y 0x00007f92474dacd0 in load_profile at ../../../src/security/security_apparmor.c:166
        silent
        if fn == 0
          cont
        p fn
        end

(gdb) bt
#0 load_profile (profile=0x7f9224049f10 "libvirt-2370eae2-cc9a-493c-b502-d2d64e2ee1d1", def=def@entry=0x7f92380e9190,
    fn=0x7f92180be390 "/var/lib/libvirt/images/eoan-disk1.snapshot2.qcow", append=append@entry=false, mgr=<optimized out>) at ../../../src/security/security_apparmor.c:166
#1 0x00007f92474db08b in AppArmorSetSecurityImageLabel (mgr=<optimized out>, def=0x7f92380e9190, src=0x7f92181f9930, flags=<optimized out>) at ../../../src/security/security_apparmor.c:830
#2 0x00007f92474d1ebe in virSecurityManagerSetImageLabel (mgr=0x7f91f401d0b0, vm=vm@entry=0x7f92380e9190, src=src@entry=0x7f92181f9930, flags=flags@entry=(unknown: 0))
    at ../../../src/security/security_manager.c:506
#3 0x00007f92474cd9a9 in virSecurityStackSetImageLabel (mgr=<optimized out>, vm=0x7f92380e9190, src=0x7f92181f9930, flags=(unknown: 0)) at ../../../src/security/security_stack.c:575
#4 0x00007f92474d1ebe in virSecurityManagerSetImageLabel (mgr=0x7f91f401b920, vm=0x7f92380e9190, src=src@entry=0x7f92181f9930, flags=flags@entry=(unknown: 0))
    at ../../../src/security/security_manager.c:506
#5 0x00007f923c27a014 in qemuSecuritySetImageLabel (driver=driver@entry=0x7f91f400f880, vm=vm@entry=0x7f92240444b0, src=src@entry=0x7f92181f9930, backingChain=backingChain@entry=false)
    at ../../../src/qemu/qemu_security.c:115
#6 0x00007f923c1dccad in qemuDomainStorageSourceAccessModify (driver=0x7f91f400f880, vm=0x7f9...

[Christian Ehrhardt  (paelzer)]

Interesting is that for:

$ virsh snapshot-create-as --domain eoan --disk-only --atomic --diskspec vda,snapshot=no --diskspec vdb,snapshot=no --diskspec vdc,file=/var/lib/libvirt/images/eoan-disk1.snapshot2.qcow,snapshot=external --diskspec vdd,file=/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow,snapshot=external

I first see:
  "/var/lib/libvirt/images/eoan-disk1.snapshot2.qcow" rwk,
and then it is gone but I see
  "/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow" rwk,

And then eventually neither of those

[Christian Ehrhardt  (paelzer)]

I think I see what happens.
virt-aa-helper works on some intermediate content, and the labelling calls only "append" something.
This works if you e.g. hot attach one and later another device.
But on this interaction with snapshots of multiple devices they seem to work on "the same" intermediate content.
It is like:

start "A"
1. result A+B
2. result A+C (totally ignoring B being added)

And eventually we only have the last disk added as apparmor rule.
Since the overall action then fails by an apparmor denial the profile is reloaded as it was before.

I need to check how/where that interim content is stored.

[Christian Ehrhardt  (paelzer)]

XML is taken via virDomainDefFormat from the contexts `virDomainDefPtr def`
That is passed to virt-aa-helper as stdin.

These calls are from the same stack.
In my example having the same object for the VM.

vm=0x7f92240444b0 which carries def=0x7f92380e9190 nto both calls and therby the same XML context it starts with (our issue).

qemuDomainSnapshotCreateDiskActive
  -> 2*qemuDomainSnapshotCreateSingleDiskActive

Since virt-aa-helper is external it doesn't know state, libvirt needs to pass that state from the labeling calls somehow.

[Christian Ehrhardt  (paelzer)]

In comparison in a hotplug case the extra disk becomes part of the ephemeral active XML.
And on the next action would be added by parsing that.
But here it renders the XML to append one, and then does the same to append the second.
But then one can hotplug multiple disks in one XML, that does work - but maybe that is serialized differently with state changes in between.

I'll recheck that later to be sure ...

[Christian Ehrhardt  (paelzer)]

virt-aa-helper has (TBH misunderstandable) call modes for appending:
 -f | --add-file <file> add file to profile
 -F | --append-file <file> append file to profile

-f does render the XML into apparmor rules and then adds the passed file (this is used atm by the labelling call).
-F keeps the apparmor rules as-is and will add a new rule at the end.

By using -F this would work for the use case in discussion (tried it manually), but I will need to consider a bunch of other interactions if we change this.

[Christian Ehrhardt  (paelzer)]

All false are reloads to restore former content (that is ok):
src/security/security_apparmor.c:706: return reload_profile(mgr, def, NULL, false);
src/security/security_apparmor.c:750: return reload_profile(mgr, def, NULL, false);
src/security/security_apparmor.c:795: return reload_profile(mgr, def, NULL, false);
src/security/security_apparmor.c:1017: return reload_profile(mgr, def, NULL, false);
src/security/security_apparmor.c:1088: return reload_profile(mgr, def, NULL, false);
src/security/security_apparmor.c:1125: return reload_profile(mgr, def, NULL, false);

All additions of paths are append=true which will cause it to use -F:
src/security/security_apparmor.c:320: return reload_profile(ptr->mgr, def, file, true);
src/security/security_apparmor.c:501: return reload_profile(mgr, def, stdin_path, true);
src/security/security_apparmor.c:733: return reload_profile(mgr, def, mem->nvdimmPath, true);
src/security/security_apparmor.c:776: return reload_profile(mgr, def, input->source.evdev, true);
src/security/security_apparmor.c:1039: ret = reload_profile(mgr, def, dev_source->data.file.path, true);
src/security/security_apparmor.c:1047: if (reload_profile(mgr, def, in, true) < 0)
src/security/security_apparmor.c:1051: if (reload_profile(mgr, def, out, true) < 0)
src/security/security_apparmor.c:1054: ret = reload_profile(mgr, def, dev_source->data.file.path, true);
src/security/security_apparmor.c:1096: return reload_profile(mgr, def, savefile, true);
src/security/security_apparmor.c:1111: rc = reload_profile(mgr, def, full_path, true);
src/security/security_apparmor.c:1114: rc = reload_profile(mgr, def, path, true);
src/security/security_apparmor.c:1152: return reload_profile(mgr, def, fd_path, true);

The only outlier to this rule is:
src/security/security_apparmor.c:466: if (load_profile(mgr, secdef->label, def, NULL, false) < 0) {

Which is what we hit in the call chain of this use-case that fails here.

[Christian Ehrhardt  (paelzer)]

TODO from here:
- Test attaching multiple disks in one XML (hot add) just to be sure
- reword the bad help text in virt-aa-helper
  - split operation modes / general options / file passing
  - add some more words to differ between -f and -F
- In AppArmorSetSecurityImageLabel for the load_profile call switch append to true
   - this is the actual fix for this issue
- AppArmorSetSecurityImageLabel can be simplified by refactoring it to use
  - reload_profile instead of load_profile
  - logic is mostly the same
- Create test builds in PPA
- Test case on test builds
- Upstream the change
- backport Fixes to affected Ubuntu release

[paelzer (paelzer-redhat-bugs)]

FYI - I was debugging this in the context of Ubuntu bug https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1845506
I think I found the root cause (see recent updates there)

The summary for now is:
- one of the labeling calls does not use append=true
- thereby the apparmor rules get re-rendered from XML throwing away former appended paths
- the snapshot case here represents two calls and the second throws away the content of the former one

If from here all goes well will submit patches some-when this week.

[Christian Ehrhardt  (paelzer)]

ok, attach-device and attach-disk only support one device on each invocation and thereby are safe even if they would call the critical code.

But these (more common) paths still pass AppArmorSetSecurityImageLabel which means if there were rules added that are not yet reflected in the XML representation they would be lost at this point.
So while not really affected it seems wise for this use-case as well to make the changes.

[Christian Ehrhardt  (paelzer)]

Example if you run snapshot and hot-add concurrently it might loose one rule and break.

[Christian Ehrhardt  (paelzer)]

I have created the patches that I had in mind and a test PPA [1] (only a subset of the patches created are in there) to verify them.
I have run test builds on Eoan and on upstream/master code levels, upstream code also ran the sysntax/style check scripts of the project.

For me the test PPA [1] works for what was identified but has further issues.
I now see (interim state) both rules there:

root@e:~# cat /etc/apparmor.d/libvirt/libvirt-2370eae2-cc9a-493c-b502-d2d64e2ee1d1.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
  "/var/log/libvirt/**/eoan.log" w,
...
  "/dev/vhost-net" rw,
  "/var/lib/libvirt/images/eoan-disk1.snapshot2.qcow" rwk,
  "/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow" rwk,

But still the snapshot fails and access still is denied by apparmor:
apparmor="DENIED" operation="open" name="/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow" requested_mask="r" ...
... name="/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow"
... name="/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow"

Hmm, those are the same paths ...
So I need to find another issue that affects this before I can go for upstreaming as I want to show the case now working ...

[1]: https://launchpad.net/~paelzer/+archive/ubuntu/bug-1845506-multi-snapshot-apparmor

[Christian Ehrhardt  (paelzer)]

By single stepping the code I see that the Denies clearly come in after the second reload_profile added the rule - so it just should not happen?

Maybe the rules aren't reloaded yet?
But I see the reload activities in dmesg:
 ... apparmor="STATUS" operation="profile_replace" ...

A manual reload while in this state shows:
  info="same as current profile, skipping"

So it clearly is loaded correctly.

Analyzing the stack on that shows that it gets back up to "qemuDomainSnapshotCreateDiskActive" without triggering the issue.
At this point it looped over the involved disks and called qemuDomainSnapshotCreateSingleDiskActive which led to the adding of the profile rules.

Then there it starts a async monitor job.
And the sync on that then triggers the denies.
  ret = qemuMonitorTransaction(priv->mon, &actions);

We know this is the qemu driver and the domain pointer is my domain, the async flag is: QEMU_ASYNC_JOB_SNAPSHOT

This command is
a) what triggers the denials
b) failing and due to that later invoking the restore of the original profile

What is submitted is the actual snapshot command to qemu:
in qemuMonitorJSONCommandWithFd:
(gdb) printf "%s\n", cmdbuf.content
{
  "execute": "transaction",
  "arguments": {
    "actions": [
      {
        "type": "blockdev-snapshot-sync",
        "data": {
          "device": "drive-virtio-disk2",
          "snapshot-file": "/var/lib/libvirt/images/eoan-disk1.snapshot2.qcow",
          "format": "qcow2"
        }
      },
      {
        "type": "blockdev-snapshot-sync",
        "data": {
          "device": "drive-virtio-disk3",
          "snapshot-file": "/var/lib/libvirt/images/eoan-disk2.snapshot1.qcow",
          "format": "qcow2"
        }
      }
    ]
  },
  "id": "libvirt-32"
}

Hmm, I'm struggling here:
1. the profile with proper rules is loaded
2. the rule is 'rwk'
3. the denial is for read which 'r' should cover
4. the paths of rule and denial are identical
5. the reported profile that blocks is the profile that we extended

I'm missing something ...

[Christian Ehrhardt  (paelzer)]

Ok the extra issue I had was only due to my stacked container setup, something broke on the apparmor nesting.
On a bare metal system with the patches applied snapshotting multiple disks at once worked fine now.

The good part of that is that I can now submit my patch series.
Done here:
 https://www.redhat.com/archives/libvir-list/2019-October/msg01002.html

Now lets wait what the upstream feedback on those will be before considering backports.
Also Eoan being released soon will most likely block uploads for a few days until 20.04 is really open.

[paelzer (paelzer-redhat-bugs)]

FYI a fix to this is on the mailing list since a few days with no response yet:
  https://www.redhat.com/archives/libvir-list/2019-October/msg01002.html

Worth a FYI ping here anyway, and maybe it is seen by that.

P.S. With the massive glib changes ongoing we might need slight adaptions depending on the order they and, but that seems to be search/replace and should be ok.

[Christian Ehrhardt  (paelzer)]

FYI - pushed in upstream git now and will be in the next release.

[Andreas Hasenack (ahasenack)]

Just adding a comment to refresh our "60 days timeout" timer, as this bug was not dropped and is being worked on.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 6.0.0-0ubuntu1

---------------
libvirt (6.0.0-0ubuntu1) focal; urgency=medium

  * Merged with Debian 5.6.0-4 from experimental and v6.0.0 from upstream
    Among many other new features and fixes this includes fixes for:
    - LP: #1859253 - rbd driver fails to create a new volume
    - LP: #1858341 - rbd driver does not list all volumes in pool
    - LP: #1845506 - Libvirt snapshot doesn't update apparmor profile
    - LP: #1854653 - slow libvirt-guests.sh during shutdown if service is off
    - LP: #1848229 - enable ppc64el to use ccf-assist feature
    - LP: #1853315 - Enable CPU Model Comparison and Baselining on s390x
    - LP: #1853317 - CCW IPL support to boot from ECKD DASDs
    - LP: #1859506 - security: AppArmor profile fixes for swtpm
    Remaining changes:
    - Disable libssh2 support (universe dependency)
    - Disable firewalld support (universe dependency)
    - Set qemu-group to kvm (for compat with older ubuntu)
    - Additional apport package-hook
    - Autostart default bridged network (As upstream does, but not Debian).
      In addition to just enabling it our solution provides:
      + do not autostart if subnet is already taken (e.g. in guests).
      + iterate some alternative subnets before giving up
    - d/p/ubuntu/Allow-libvirt-group-to-access-the-socket.patch: This is
      the group based access to libvirt functions as it was used in Ubuntu
      for quite long.
      + d/p/ubuntu/daemon-augeas-fix-expected.patch fix some related tests
        due to the group access change.
      + d/libvirt-daemon-system.postinst: add users in sudo to the libvirt
        group.
    - ubuntu/parallel-shutdown.patch: set parallel shutdown by default.
    - Update Vcs-Git and Vcs-Browser fields to point to launchpad
    - Update README.Debian with Ubuntu changes
    - Enable some additional features on ppc64el and s390x (for arch parity)
      + systemtap, zfs, numa and numad on s390x.
      + systemtap on ppc64el.
    - d/p/ubuntu/ubuntu_machine_type.patch: accept ubuntu types as pci440fx
    - Further upstreamed apparmor Delta, especially any new one
      Our former delta is split into logical pieces and is either Ubuntu only
      or is part of a continuous upstreaming effort.
      Listing related remaining changes in debian/patches/ubuntu-aa/:
    - fix autopkgtests
      + d/t/control, d/t/smoke-qemu-session: fixup smoke-qemu-session by making
        vmlinuz available and accessible (Debian bug 848314)
      + d/t/control: fix smoke-qemu-session by ensuring the service will run
        installing libvirt-daemon-system
      + d/t/smoke-lxc: fix smoke-lxc by ignoring potential issues on destroy as
        long as the following undefine succeeds
      + d/t/smoke-lxc: use systemd instead of sysV to restart the service
    - dnsmasq related enhancements
      + run dnsmasq as libvirt-dnsmasq (LP: 1743718)
      + d/libvirt-daemon-system.postinst: add libvirt-dnsmasq user and group
      + d/libvirt-daemon-system.postrm: remove libvirt-dnsmasq user and group
        on purge
      + d/p/ubuntu/dnsmasq-as-priv-user: write dnsmasq config with user
        libvirt-dnsmasq and adapt t...

[Christian Ehrhardt  (paelzer)]

We had a discussion about further backports in [1], but I think the answer is no unless we have a stronger reasoning. Let us mark the tags accordingly.

[1]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1892306/comments/7