nova creates an invalid ethernet/bridge interface definition in virsh xml

[Michael Henkel]

Description
===========

https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61
sets the script path of an ethernet interface to ""

https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1228
checks script for None. As it is not none but a string it adds an empty
script path to the ethernet interface definition in the virsh xml

Steps to reproduce
==================

nova generated virsh:

[root@overcloud-novacompute-0 heat-admin]# cat 2.xml |grep tap -A5 -B3
    <interface type='ethernet'>
      <mac address='02:b9:38:d2:ef:bf'/>
      <script path=''/>
      <target dev='tapb938d2ef-bf'/>
      <model type='virtio'/>
      <driver name='qemu'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

XML validation:

[root@overcloud-novacompute-0 heat-admin]# virt-xml-validate 2.xml
Relax-NG validity error : Extra element devices in interleave
2.xml:59: element devices: Relax-NG validity error : Element domain failed to validate content
2.xml fails to validate

removing the <script path=''/> element the xml validation succeeds:

[root@overcloud-novacompute-0 heat-admin]# cat 1.xml |grep tap -A5 -B2
    <interface type='ethernet'>
      <mac address='02:b9:38:d2:ef:bf'/>
      <target dev='tapb938d2ef-bf'/>
      <model type='virtio'/>
      <driver name='qemu'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
[root@overcloud-novacompute-0 heat-admin]# virt-xml-validate 1.xml
1.xml validates

Point is that libvirt <2.0.0 is more tolerant. libvirt 2.0.0 throws a segfault:

Dec 9 13:30:32 comp1 kernel: libvirtd[1048]: segfault at 8 ip 00007fc9ff09e1c3 sp 00007fc9edfef1d0 error 4 in libvirt.so.0.2000.0[7fc9fef4b000+352000]
Dec 9 13:30:32 comp1 journal: End of file while reading data: Input/output error
Dec 9 13:30:32 comp1 systemd: libvirtd.service: main process exited, code=killed, status=11/SEGV
Dec 9 13:30:32 comp1 systemd: Unit libvirtd.service entered failed state.
Dec 9 13:30:32 comp1 systemd: libvirtd.service failed.
Dec 9 13:30:32 comp1 systemd: libvirtd.service holdoff time over, scheduling restart.
Dec 9 13:30:32 comp1 systemd: Starting Virtualization daemon...
Dec 9 13:30:32 comp1 systemd: Started Virtualization daemon.

Expected result
===============
VM can be started
instead of checking for None, config.py should check for an empty string before
adding script path

Actual result
=============
VM doesn't start

Environment
===========
OSP10/Newton, libvirt 2.0.0

[Stephen Gordon (sgordon)]

In addition, there is the complication that operators now have many existing VMs which have been created with the script path="" directive that Libvirt will refuse to recognize.

[Nell Jerram (neil-jerram)]

Looking forwards, I'm pretty sure the right thing is to set conf.script = None instead of = '', so I've posted a change for that at https://review.openstack.org/#/c/411936/.

@sgordon - For the existing VMs that you mention, is there anything that can be done other than fixing libvirt to be back-compatible, in the sense of ignoring a script path of '' ?

[Stephen Gordon (sgordon)]

Marked confirmed as I'm aware of at least two other users reporting this or related symptoms of it on the M/L.

[Stephen Gordon (sgordon)]

See also: http://lists.openstack.org/pipermail/openstack-dev/2016-December/109401.html

[YuYang (yuyangwang1985)]

hi:i change the code by you ,but still get the error:
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host [req-9f0f7a12-5947-4d8d-be12-904376ab13cc f78cc871fb804d548b956ca8971dbf32 500ef1d3e27a410daad8075fd915087e - - -] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host Traceback (most recent call last):
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 411, in get_connection
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host conn = self._get_connection()
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 398, in _get_connection
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host wrapped_conn = self._get_new_connection()
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 350, in _get_new_connection
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host wrapped_conn = self._connect(self._uri, self._read_only)
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 204, in _connect
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host libvirt.openAuth, uri, auth, flags)
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host rv = execute(f, *args, **kwargs)
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host six.reraise(c, e, tb)
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host rv = meth(*args, **kwargs)
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host File "/usr/lib64/python2.7/site-packages/libvirt.py", line 105, in openAuth
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2016-12-29 05:14:24.367 2371 ERROR nova.virt.libvirt.host
2016-12-29 05:14:24.371 2371 WARNING nova.compute.manager [req-9f0f7a12-5947-4d8d-be12-904376ab13cc f78cc871fb804d548b956ca8971dbf32 500ef1d3e27a410daad8075fd915087e - - -] Could not clean up failed build, not rescheduling. Error: Connection to the hypervisor is broken on host: OS-N-Compute-1
2016-12-29 05:14:24.729 2371 ERROR nova.compute.manager [req-9f0f7a12-5947-4d8d-be12-904376ab13cc f78cc871fb804d548b956ca8971dbf32 500ef1d3e27a410daad8075fd915087e - - -] [instance: b40d05c9-5616-433b-9333-bca270f5873e] Build of instance b40d05c9-5616-433b-9333-bca270f5873e abort...

[YuYang (yuyangwang1985)]

my environment:
libvirt-daemon-driver-qemu-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-kvm-2.0.0-10.el7_3.2.x86_64
libvirt-client-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-storage-2.0.0-10.el7_3.2.x86_64
libvirt-python-2.0.0-2.el7.x86_64
libvirt-daemon-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-secret-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-interface-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-network-2.0.0-10.el7_3.2.x86_64
libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.2.x86_64

CentOS Linux release 7.3.1611 (Core)

[Michael Henkel (mhenkel-3)]

did you restart nova-compute?

> Am 29.12.2016 um 11:31 schrieb YuYang <email address hidden>:
>
> my environment:
> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-kvm-2.0.0-10.el7_3.2.x86_64
> libvirt-client-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-storage-2.0.0-10.el7_3.2.x86_64
> libvirt-python-2.0.0-2.el7.x86_64
> libvirt-daemon-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-secret-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-interface-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-network-2.0.0-10.el7_3.2.x86_64
> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.2.x86_64
>
> CentOS Linux release 7.3.1611 (Core)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1649527
>
> Title:
> nova creates an invalid ethernet/bridge interface definition in virsh
> xml
>
> Status in OpenStack Compute (nova):
> Confirmed
>
> Bug description:
> Description
> ===========
>
> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61
> sets the script path of an ethernet interface to ""
>
> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1228
> checks script for None. As it is not none but a string it adds an empty
> script path to the ethernet interface definition in the virsh xml
>
> Steps to reproduce
> ==================
>
> nova generated virsh:
>
> [root@overcloud-novacompute-0 heat-admin]# cat 2.xml |grep tap -A5 -B3
> <interface type='ethernet'>
> <mac address='02:b9:38:d2:ef:bf'/>
> <script path=''/>
> <target dev='tapb938d2ef-bf'/>
> <model type='virtio'/>
> <driver name='qemu'/>
> <alias name='net0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> </interface>
>
> XML validation:
>
> [root@overcloud-novacompute-0 heat-admin]# virt-xml-validate 2.xml
> Relax-NG validity error : Extra element devices in interleave
> 2.xml:59: element devices: Relax-NG validity error : Element domain failed to validate content
> 2.xml fails to validate
>
> removing the <script path=''/> element the xml validation succeeds:
>
> [root@overcloud-novacompute-0 heat-admin]# cat 1.xml |grep tap -A5 -B2
> <interface type='ethernet'>
> <mac address='02:b9:38:d2:ef:bf'/>
> <target dev='tapb938d2ef-bf'/>
> <model type='virtio'/>
> <driver name='qemu'/>
> <alias name='net0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> </interface>
> [root@overcloud-novacompute-0 heat-admin]# virt-xml-validate 1.xml
> 1.xml validates
>
> Point is that libvirt <2.0.0 is more tolerant. libvirt 2.0.0 throws a segfault:
>
> Dec 9 13:30:32 comp1 kernel: libvirtd[1048]: segfault at 8 ip 00007fc9ff09e1c3 sp 00007fc9edfef1d0 error 4 in libvirt.so.0.2000.0[7fc9fef4b000+352000]
> Dec 9 13:30:32 comp1 journal: End of file while reading data: Input/output error
> Dec 9 13:30:32 comp1 systemd: libvirtd.service: main process exited, code=killed, status=11/SEGV
> Dec 9 13:30:...

[Xizhen Du (xizhendu)]

Having the exact same error with Newton on centos 7.3

# Systems log
libvirtd.service: main process exited, code=killed, status=11/SEGV
Unit libvirtd.service entered failed state.
libvirtd.service failed.
libvirtd.service holdoff time over, scheduling restart.
Starting Virtualization daemon...
Started Virtualization daemon.

# And nova-compute.log:
The exact same with comment #6 by YuYang (yuyangwang1985).

[YuYang (yuyangwang1985)]

hi Michael Henkel :i retstart nova-compute and reboot the system,also get the error!

[Xizhen Du (xizhendu)]

Libvirtd gets crashed whenever I tried to boot an instance within openstack/newton, both from dashboard or openstack cli.

The compute host itself certainly can boot a guest successfully if I use virt-install tools.

Attached a latest crash log sent to root.

reason: libvirtd killed by SIGSEGV
cmdline: /usr/sbin/libvirtd
executable: /usr/sbin/libvirtd
package: libvirt-daemon-2.0.0-10.el7_3.2
component: libvirt

environ:
:LANG=en_GB.UTF-8
:PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
:NOTIFY_SOCKET=/run/systemd/notify
exploitable:
:Likely crash reason: Jump to an invalid address
:Exploitable rating (0-9 scale): 6

[YuYang (yuyangwang1985)]

i downgrade the libvert version to 1.2.17 is ok!

[root@OS-N-Compute-1 ~]# virsh version
Compiled against library: libvirt 1.2.17
Using library: libvirt 1.2.17
Using API: QEMU 1.2.17
Running hypervisor: QEMU 2.6.0

[YuYang (yuyangwang1985)]

[root@OS-N-Compute-1 ~]# rpm -qa | grep libvirt
libvirt-daemon-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-storage-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-kvm-1.2.17-13.el7_2.5.x86_64
libvirt-client-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-network-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-secret-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-interface-1.2.17-13.el7_2.5.x86_64
libvirt-python-1.2.17-2.el7.x86_64
libvirt-daemon-driver-qemu-1.2.17-13.el7_2.5.x86_64
libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.5.x86_64

[Xizhen Du (xizhendu)]

To my case it is related the cpu_model parameter in nova.conf.
Nova generates the file with 'cpu_model' for an instance, and libvirt crashes if not proper model defined in xml.

[Antonio Ojea (aojea)]

We are facing this issue with Centos 7.3 and midonet, using Mitaka and Newton

2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] six.reraise(self.type_, self.value, self.tb)
    </video>
    <memballoon model='virtio'>
      <stats period='10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [req-ee0ff712-fa23-4c92-8f13-cd2317a22d18 d9b418abe271405c9c74a45169813769 a8fc3474d5324dbeb58db66f140
d2402 - - -] [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] Instance failed to spawn
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] Traceback (most recent call last):
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/compute
/manager.py", line 2218, in _build_resources
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] yield resources
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/compute
/manager.py", line 2064, in _build_and_run_instance
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] block_device_info=block_device_info)
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/virt/li
bvirt/driver.py", line 2779, in spawn
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] block_device_info=block_device_info)
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/virt/li
bvirt/driver.py", line 4923, in _create_domain_and_network
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] xml, pause=pause, power_on=power_on)
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/virt/li
bvirt/driver.py", line 4854, in _create_domain
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] guest.launch(pause=pause)
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/nova/virt/li
bvirt/guest.py", line 142, in launch
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] self._encoded_xml, errors='ignore')
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183545d2b] File "/usr/lib/python2.7/site-packages/oslo_utils/e
xcutils.py", line 220, in __exit__
2017-01-05 22:05:15.984 10009 ERROR nova.compute.manager [instance: d160e467-2dca-46aa-ba41-1bc183...

[Antonio Ojea (aojea)]

The patch is blocked with a -1, but I can't understand what's needed to solve the issue.

Can it be fixed in nova or should it be fixed in other project?

[Nell Jerram (neil-jerram)]

@A.Ojea - Matt Riedemann asked a good question about upgrading from before to after the fix, which I need to think about and answer. It's on my list! (But also happy in case someone else wants to answer for me!)

[Nell Jerram (neil-jerram)]

OK, I just answered Matt's question in the review - please let me know what you think. (CC @sgordon)

[Stephen Gordon (sgordon)]

Neil is still the owner here, I just appended a release note to his patch.

[Aleksandr Didenko (adidenko)]

I have the same problem on Ubuntu 16.04, stable/newton nova, libvirt-2.5.0 and networking-calico neutron plugin. Also I've tested the Neil's patch and can confirm it fixes the issue and I'm able to run VMs. JFYI.

[Nell Jerram (neil-jerram)]

@adidenko: Ooh, interesting, do you know if Xenial has recently upgraded libvirt? I'm wondering why I didn't notice this problem before in my testing on Xenial?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/411936
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=847952927c60ed0577bc835adf607ed7b8f15240
Submitter: Jenkins
Branch: master

commit 847952927c60ed0577bc835adf607ed7b8f15240
Author: Neil Jerram <email address hidden>
Date: Fri Dec 16 17:49:59 2016 +0000

    libvirt: avoid generating script with empty path

    Previously, libvirt just appended 'script=' onto the QEMU cmd line
    according to what <script path=''/> contained, letting QEMU execute the
    script. That was flawed from security POV (you don't want QEMU to be
    allowed to execute anything), so newer libvirt (as of [1]) executes the
    script now. But the libvirt code doesn't allow this corner case (of
    allowing and ignoring an empty script path) whereas apparently the QEMU
    code does.

    So the Nova setting of '' used to work by accident, but now does not.

    [1]
    http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=9c17d66 (autocreate
    tap device for ethernet network type)

    Closes-Bug: #1649527
    Change-Id: I4f97c05e2dec610af22a5150dd27696e1d767896

[Aleksandr Didenko (adidenko)]

@neil-jerram, sorry, forgot to mention that I'm running libvirt in Debian container on Ubuntu host system. So it's not default Ubuntu libvirt package.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/425637

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/425637
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=99f8a3c4e9d903d48e5c7e245bcb2d3299b7904d
Submitter: Jenkins
Branch: stable/newton

commit 99f8a3c4e9d903d48e5c7e245bcb2d3299b7904d
Author: Neil Jerram <email address hidden>
Date: Fri Dec 16 17:49:59 2016 +0000

    libvirt: avoid generating script with empty path

    Previously, libvirt just appended 'script=' onto the QEMU cmd line
    according to what <script path=''/> contained, letting QEMU execute the
    script. That was flawed from security POV (you don't want QEMU to be
    allowed to execute anything), so newer libvirt (as of [1]) executes the
    script now. But the libvirt code doesn't allow this corner case (of
    allowing and ignoring an empty script path) whereas apparently the QEMU
    code does.

    So the Nova setting of '' used to work by accident, but now does not.

    [1]
    http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=9c17d66 (autocreate
    tap device for ethernet network type)

    Closes-Bug: #1649527
    Change-Id: I4f97c05e2dec610af22a5150dd27696e1d767896
    (cherry picked from commit 847952927c60ed0577bc835adf607ed7b8f15240)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/431570

[Assen Tarlov (atarlov)]

The same issue appears in OSP8/Liberty - RHEL7.3

By deploying instances in the Compute Nodes

 [heat-admin@overcloud-compute-1 ~]$ libvirtd –version
   libvirtd (libvirt) 2.0.0

/var/log/nova-compute.

- ERROR nova.compute.manager [instance: b85cd706-bb1e-4daa-8438-8481ade095d6] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
   ERROR nova.compute.manager [instance: b85cd706-bb1e-4daa-8438-8481ade095d6] libvirtError: Cannot find '' in path: No such file or directory

[heat-admin@overcloud-compute-1 ~]$ cat /usr/lib/python2.7/site-packages/nova/virt/libvirt/designer.py | grep conf.script -A5 -B3

    conf.net_type = "ethernet"
    conf.target_dev = tapname
    conf.script = ""

[Logan V (loganv)]

On testing the Newton commit, I'm seeing the issue described here: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1665698

Based on the wording of https://en.wikibooks.org/wiki/QEMU/Networking, I think by removing the <script path=''/>, qemu is reverting to a default of trying to run '/etc/qemu-ifup', which is not allowed by the Ubuntu apparmor shipped with libvirt.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 14.0.4 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Antonio Ojea (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/431570
Reason: Mitaka is EOL

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/448203

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/448242

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/448253

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/448203
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a41d265a19b7bcb1af8fc179bf864e00023c6cc6
Submitter: Jenkins
Branch: master

commit a41d265a19b7bcb1af8fc179bf864e00023c6cc6
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 21 13:18:08 2017 -0400

    libvirt: conditionally set script path for ethernet vif types

    Change I4f97c05e2dec610af22a5150dd27696e1d767896 worked around
    a change introduced in libvirt 1.3.3 where the script path on
    a LibvirtConfigGuestInterface could not be the emptry string
    because libvirt would literally take that as the path and couldn't
    resolve it, when in fact it used to indicate to libvirt that the
    script path is a noop. This has been fixed in libvirt 3.1.

    On Ubuntu with libvirt<1.3.3, if the script path is None then
    it defaults to /etc/qemu-ifup which is blocked by AppArmor.

    So this change adds a conditional check when setting the script
    path value based on the libvirt version so we can straddle releases.

    Change-Id: I192c61b93bd3736fdfe16b6a6906d58997d3eef9
    Closes-Bug: #1665698
    Related-Bug: #1649527

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/448242
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cc495a24656893c94031f491a3fed2bc94fe1850
Submitter: Jenkins
Branch: stable/ocata

commit cc495a24656893c94031f491a3fed2bc94fe1850
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 21 13:18:08 2017 -0400

    libvirt: conditionally set script path for ethernet vif types

    Change I4f97c05e2dec610af22a5150dd27696e1d767896 worked around
    a change introduced in libvirt 1.3.3 where the script path on
    a LibvirtConfigGuestInterface could not be the emptry string
    because libvirt would literally take that as the path and couldn't
    resolve it, when in fact it used to indicate to libvirt that the
    script path is a noop. This has been fixed in libvirt 3.1.

    On Ubuntu with libvirt<1.3.3, if the script path is None then
    it defaults to /etc/qemu-ifup which is blocked by AppArmor.

    So this change adds a conditional check when setting the script
    path value based on the libvirt version so we can straddle releases.

    Change-Id: I192c61b93bd3736fdfe16b6a6906d58997d3eef9
    Closes-Bug: #1665698
    Related-Bug: #1649527
    (cherry picked from commit a41d265a19b7bcb1af8fc179bf864e00023c6cc6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/448253
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8142d526dfd6f4a56dbe382d25cf4110abf57f44
Submitter: Jenkins
Branch: stable/newton

commit 8142d526dfd6f4a56dbe382d25cf4110abf57f44
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 21 13:18:08 2017 -0400

    libvirt: conditionally set script path for ethernet vif types

    Change I4f97c05e2dec610af22a5150dd27696e1d767896 worked around
    a change introduced in libvirt 1.3.3 where the script path on
    a LibvirtConfigGuestInterface could not be the emptry string
    because libvirt would literally take that as the path and couldn't
    resolve it, when in fact it used to indicate to libvirt that the
    script path is a noop. This has been fixed in libvirt 3.1.

    On Ubuntu with libvirt<1.3.3, if the script path is None then
    it defaults to /etc/qemu-ifup which is blocked by AppArmor.

    So this change adds a conditional check when setting the script
    path value based on the libvirt version so we can straddle releases.

    Change-Id: I192c61b93bd3736fdfe16b6a6906d58997d3eef9
    Closes-Bug: #1665698
    Related-Bug: #1649527
    (cherry picked from commit a41d265a19b7bcb1af8fc179bf864e00023c6cc6)
    (cherry picked from commit cc495a24656893c94031f491a3fed2bc94fe1850)