virt-aa-helper: do not fail with memory slots specified in guest xml

[bugproxy]

== Comment: #1 - INDIRA P. JOGA <email address hidden> - 2018-01-04 11:57:53 ==
Problem Description:
===================
Not able to start the guest on Ubuntu1804 KVM host machine

Steps to re-create:
==================
> Installed Ubuntu1804 on boslcp3 host.

root@boslcp3:/home# uname -a
Linux boslcp3 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3:/home# uname -r
4.13.0-17-generic

> Installed qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils packages

root@boslcp3:/home# virsh version
Compiled against library: libvirt 3.6.0
Using library: libvirt 3.6.0
Using API: QEMU 3.6.0
Running hypervisor: QEMU 2.10.1

> Defined the guest boslcp3g1 from host machine

root@boslcp3:/home# virsh list --all
 Id Name State
----------------------------------------------------
 - boslcp3g1 shut off

> Started the guest and it fails with ?cannot load AppArmor profile 'libvirt-95374879-0ed3-4562-a00f-e47d9aaf285c??

root@boslcp3:/home# virsh start --console boslcp3g1
error: Failed to start domain boslcp3g1
error: internal error: cannot load AppArmor profile 'libvirt-95374879-0ed3-4562-a00f-e47d9aaf285c'

> Not able to start the any of the guests from KVM host machine.

XML:
****
root@boslcp3:/home# virsh dumpxml boslcp3g1
<domain type='kvm'>
  <name>boslcp3g1</name>
  <uuid>95374879-0ed3-4562-a00f-e47d9aaf285c</uuid>
  <maxMemory slots='16' unit='KiB'>10485760</maxMemory>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='0'/>
    </hugepages>
  </memoryBacking>
  <vcpu placement='static' current='16'>32</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='ppc64le' machine='pseries-2.10'>hvm</type>
    <boot dev='hd'/>
    <boot dev='network'/>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='forbid'>power9</model>
    <topology sockets='2' cores='8' threads='2'/>
    <numa>
      <cell id='0' cpus='0-7' memory='3145728' unit='KiB'/>
      <cell id='1' cpus='8-15' memory='3145728' unit='KiB'/>
    </numa>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>coredump-restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/disk/by-id/wwn-0x600507680183050d28000000000002a4-part1'/>
      <target dev='sda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='wwn-0x600507680183050d28000000000002a4-part2'/>
      <target dev='sdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='wwn-0x600507680183050d28000000000002a4-part3'/>
      <target dev='sdc' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <model name='spapr-pci-host-bridge'/>
      <target index='0'/>
    </controller>
    <controller type='pci' index='1' model='pci-root'>
      <model name='spapr-pci-host-bridge'/>
      <target index='1'/>
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:e5:81:1f'/>
      <source dev='enP2p1s0f0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </memballoon>
    <panic model='pseries'/>
  </devices>
</domain>

== Comment: #3 - INDIRA P. JOGA <email address hidden> - 2018-01-05 00:06:42 ==
This is blocking the test execution.

Regards,
Indira

== Comment: #5 - INDIRA P. JOGA <email address hidden> - 2018-01-05 00:41:19 ==
> Collected 1717970.apport log

root@boslcp3:/tmp# sudo apport-cli -f --save /tmp/1717970.apport -p linux-image-4.13.0-17-generic

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
.....tar: Removing leading `/' from member names
....tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: /var/log/opal-elog: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
........................................................................................................................................................................................................................................................................

> root@boslcp3:/tmp# ls -l 1717970.apport
-rw-r--r-- 1 root root 1941570 Jan 4 17:35 1717970.apport

> Attached /tmp/1717970.apport log

Other info:
***********
root@boslcp3:~# sudo grep -Hi uuid /etc/libvirt/qemu/*.xml
/etc/libvirt/qemu/boslcp3g1.xml: <uuid>95374879-0ed3-4562-a00f-e47d9aaf285c</uuid>

root@boslcp3:~# ls -l /etc/apparmor.d/libvirt
total 8
-rw-r--r-- 1 root root 314 Oct 24 07:30 TEMPLATE.lxc
-rw-r--r-- 1 root root 164 Oct 24 07:30 TEMPLATE.qemu

== Comment: #6 - INDIRA P. JOGA <email address hidden> - 2018-01-05 00:49:43 ==

== Comment: #10 - Shivaprasad G. Bhat <email address hidden> - 2018-01-05 02:53:44 ==
apparmor-profiles package was missing. Now, the guest is staring. Let me know if there is more to this bug or we can close it

== Comment: #12 - INDIRA P. JOGA <email address hidden> - 2018-01-07 13:35:32 ==
> Reopening the bug as i am still facing issue while starting the other guest boslcp3g2 from boslcp3 host machine as below.

root@boslcp3:~# virsh start --console boslcp3g2
error: Failed to start domain boslcp3g2
error: internal error: cannot load AppArmor profile 'libvirt-a486c1fd-7c5b-4f4c-8fc1-843d7f63969d'

> checked for kern.log

root@boslcp3:~# grep virt /var/log/kern.log
root@boslcp3:~#

> Also checked the apparmor.service which is active

root@boslcp3:~# systemctl status apparmor.service
? apparmor.service - AppArmor initialization
   Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sun 2018-01-07 01:15:17 CST; 43min ago
     Docs: man:apparmor(7)
           http://wiki.apparmor.net/
  Process: 6239 ExecReload=/etc/init.d/apparmor reload (code=exited, status=0/SUCCESS)
 Main PID: 1576 (code=exited, status=0/SUCCESS)

> Also checked for apparmor-profiles packages and it shows

root@boslcp3:~# dpkg --list "apparmor"*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================================-========================-========================-===================================================================================
ii apparmor 2.11.0-2ubuntu18 ppc64el user-space parser utility for AppArmor
ii apparmor-profiles 2.11.0-2ubuntu18 all profiles for AppArmor Security policies
un apparmor-profiles-extra <none> <none> (no description available)
ii apparmor-utils 2.11.0-2ubuntu18 ppc64el utilities for controlling AppArmor

> Unable to start the guest as it fails with cannot load AppArmor profile 'libvirt-a486c1fd-7c5b-4f4c-8fc1-843d7f63969d' error.

Please let me know if anything else missing to start the guest on KVM host machine

Regards,
Indira

== Comment: #13 - INDIRA P. JOGA <email address hidden> - 2018-01-07 23:19:32 ==
Attaching the boslcp3g2 guest xml below

root@boslcp3:~# virsh dumpxml boslcp3g2
<domain type='kvm'>
  <name>boslcp3g2</name>
  <uuid>a486c1fd-7c5b-4f4c-8fc1-843d7f63969d</uuid>
  <maxMemory slots='16' unit='KiB'>16492674416640</maxMemory>
  <memory unit='KiB'>13194139533312</memory>
  <currentMemory unit='KiB'>13194139533312</currentMemory>
  <vcpu placement='static' current='16'>32</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='ppc64le' machine='pseries-2.10'>hvm</type>
    <boot dev='hd'/>
    <boot dev='network'/>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='forbid'>power9</model>
    <topology sockets='8' cores='4' threads='1'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>coredump-restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/boslcp3g2_pool/boslcp3g2_root'/>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/boslcp3g2_pool/boslcp3g2_io1'/>
      <target dev='sdb' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/boslcp3g2_pool/boslcp3g2_io2'/>
      <target dev='sdc' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <model name='spapr-pci-host-bridge'/>
      <target index='0'/>
    </controller>
    <controller type='pci' index='1' model='pci-root'>
      <model name='spapr-pci-host-bridge'/>
      <target index='1'/>
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:42:3e:fa'/>
      <source dev='enP2p1s0f0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </memballoon>
    <panic model='pseries'/>
  </devices>
</domain>
>
</domain>

Regards,
Indira

== Comment: #17 - Shivaprasad G. Bhat <email address hidden> - 2018-01-11 04:17:20 ==
Posted patch upstream for review.

https://www.redhat.com/archives/libvir-list/2018-January/msg00339.html

Regards,
Indira

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[Christian Ehrhardt  (paelzer)]

Bionic will have what currently is in [1] right now.
I'd ask you to test with that if possible.

Also to be sure since I successfully started hundreds of guests for testing on P8 recently.
Is this again on P9 like (can't find the other bug atm :-/)?

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3108/

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-01 23:52 EDT-------
This has got nothing to do with P8/P9/hardware specific. Basically features like cpu and memory hotplug add extra xml tags which are not parsable when used with virt-aa-helper as the flags are not set. virt-aa-helper fails to create a profile because of that.

[Nish Aravamudan (nacc)]

It would appear that upstream feedback resulted in https://www.redhat.com/archives/libvir-list/2018-January/msg00615.html

[Christian Ehrhardt  (paelzer)]

Sorry I missed the initial mailing list link in all the initial text :-/
It was just too similar to another bug that was coming by last week that I short-cut the wrong way.

Thanks Shivaprasad for clarifications.
Also thansk Nish to spot there is a new revision.

With a new look I recreated the case.
The essential addition to trigger is (independent to the actual values):
<maxMemory slots='16' unit='KiB'>10485760</maxMemory>
(nothing more of the xml needs to be special)

Code Review:
We can't link against qemu_domain.c, but a comment that the define is essentially taken from virQEMUDriverDomainDefParserConfig might be nice to later on follow any updates there.
Also mentioning why you dropped VIR_DOMAIN_DEF_FEATURE_USER_ALIAS for the same reason.
(I'll reply so to the list shortly)

The error can also be triggered in an isolated way.
Get the uuid of the guest and dump it to a test.xml file.
Then run:
/usr/lib/libvirt/virt-aa-helper --create --dryrun --uuid 'libvirt-e137cfac-8880-484a-95f5-205a9fd604e7' < test.xml
virt-aa-helper: error: could not parse XML
virt-aa-helper: error: could not get VM definition

I built a test in a ppa at [1] with your patch.
With that installed retesting the above it works well.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3122

[Christian Ehrhardt  (paelzer)]

I replied on the mailing list.

I also took a reminder to pick this up once upstream.
I'll later on (before 18.04 release) pick a collection of fixes for libvirt to stabilize it further - this change will be part of it.

[Christian Ehrhardt  (paelzer)]

respin of the build available in the same ppa.
(Including the final fix as committed upstream)

[Christian Ehrhardt  (paelzer)]

LGTM in regression checks, uploading ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 4.0.0-1ubuntu2

---------------
libvirt (4.0.0-1ubuntu2) bionic; urgency=medium

  * d/p/ubuntu-aa/0025-apparmor-fix-newer-virt-manager-1.4.0.patch: refreshed
    as libvirt 4.0 needs a reversed rule for openGraphicsFD (LP: #1747442)
    - refreshed 0032 and 0040 to match the new context.
  * d/p/ubuntu/virt-aa-helper-Set-the-supported-features.patch: allow parsing
    of memory slots and other extended features without breaking
    virt-aa-helper (LP: #1746431).

 -- Christian Ehrhardt <email address hidden> Fri, 02 Feb 2018 07:31:17 +0100

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-20 06:37 EDT-------
Is this fix available in official daily ubuntu1804 daily build?

[Christian Ehrhardt  (paelzer)]

yes - the comment above lists the package version that incldued the fix.
YOu can either upgrade to that or use a daily build that includes it.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-21 06:12 EDT-------
Hi,

--> Able to start the guests from host system with max memory tag successfully with libvirt 4.0.0 version. Issue fixed with latest 4.0.0 version.

# virsh version
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.0

root@boslcp3:/home# uname -a
Linux boslcp3 4.15.0-041500rc9-generic #201801212130 SMP Mon Jan 22 03:36:42 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3:/home# uname -r
4.15.0-041500rc9-generic

root@boslcp3:/home# virsh dumpxml boslcp3g1 | grep max
<maxMemory slots='16' unit='KiB'>16777216</maxMemory>

root@boslcp3:/home# virsh start --console boslcp3g1
Domain boslcp3g1 started
Connected to domain boslcp3g1
Escape character is ^]
*****
QEMU Starting
Build Date = Dec 13 2017 13:46:58
FW Version = buildd@ release 20170724
Press "s" to enter Open Firmware.

Press F12 for boot menu.

root@boslcp3g1:~# free -h
total used free shared buff/cache available
Mem: 6.0G 428M 4.4G 7.6M 1.1G 4.6G
Swap: 2.0G 0B 2.0G