Got following error log when booting instance.
Environment: centos+source+vlan
# nova-compute.log
2017-02-01 23:33:04.945 7 DEBUG oslo_concurrency.lockutils [req-6577d10d-0f06-4f2a-bbe1-3c0a6391195e - - - - -] Releasing semaphore "refresh_cache-4c80aaf5-7935-46da-97a6-a46dd77eee03" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
2017-02-01 23:33:05.169 7 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: cf6b9559639f4394a71c1633c06ebe40 __call__ /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:299
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [req-b355693b-f654-4c15-a364-d8acc34ac180 - - - - -] [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] Instance failed to spawn
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] Traceback (most recent call last):
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 2119, in _build_resources
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] yield resources
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 1924, in _build_and_run_instance
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] block_device_info=block_device_info)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2684, in spawn
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] post_xml_callback=gen_confdrive)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5083, in _create_domain_and_network
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] block_device_info, guest)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self.force_reraise()
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(self.type_, self.value, self.tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5058, in _create_domain_and_network
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] post_xml_callback=post_xml_callback)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4978, in _create_domain
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] guest.launch(pause=pause)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 145, in launch
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self._encoded_xml, errors='ignore')
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self.force_reraise()
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(self.type_, self.value, self.tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 140, in launch
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] return self._domain.createWithFlags(flags)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] rv = execute(f, *args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(c, e, tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] rv = meth(*args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/libvirt.py", line 1065, in createWithFlags
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] libvirtError: internal error: Unable to add port tapb5ee7be4-49 to OVS bridge br-int
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03]
# libvirtd.log
2017-02-02 04:33:04.815+0000: 960: info : libvirt version: 2.0.0, package: 10.el7_3.4 (CentOS BuildSystem <http://bugs.centos.org>, 2017-01-17-23:37:48, c1bm.rdu2.centos.org)
2017-02-02 04:33:04.815+0000: 960: info : hostname: aio
2017-02-02 04:33:04.815+0000: 960: error : virExec:492 : Cannot find 'ovs-vsctl' in path: No such file or directory
2017-02-02 04:33:04.815+0000: 960: error : virNetDevOpenvswitchAddPort:155 : internal error: Unable to add port tapb5ee7be4-49 to OVS bridge br-int
2017-02-02 04:33:04.838+0000: 960: error : virExec:492 : Cannot find 'ovs-vsctl' in path: No such file or directory
2017-02-02 04:33:04.838+0000: 960: error : virNetDevOpenvswitchRemovePort:188 : internal error: Unable to delete port tapb5ee7be4-49 from OVS
after checked the libvirt source code, libvirtd container need openvswitch package [1]
[1] https://github.com/libvirt/libvirt/blob/master/src/util/virnetdevopenvswitch.c#L90
Fix proposed to branch: master /review. openstack. org/428001
Review: https:/