kolla

libvirt container require openvswitch package

Bug #1661152 reported by Jeffrey Zhang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
High
Jeffrey Zhang
kolla ocata-rc1 "rc1"

Bug Description

Got following error log when booting instance.

Environment: centos+source+vlan

# nova-compute.log
2017-02-01 23:33:04.945 7 DEBUG oslo_concurrency.lockutils [req-6577d10d-0f06-4f2a-bbe1-3c0a6391195e - - - - -] Releasing semaphore "refresh_cache-4c80aaf5-7935-46da-97a6-a46dd77eee03" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
2017-02-01 23:33:05.169 7 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: cf6b9559639f4394a71c1633c06ebe40 __call__ /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:299
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [req-b355693b-f654-4c15-a364-d8acc34ac180 - - - - -] [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] Instance failed to spawn
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] Traceback (most recent call last):
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 2119, in _build_resources
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] yield resources
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 1924, in _build_and_run_instance
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] block_device_info=block_device_info)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2684, in spawn
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] post_xml_callback=gen_confdrive)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5083, in _create_domain_and_network
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] block_device_info, guest)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self.force_reraise()
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(self.type_, self.value, self.tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5058, in _create_domain_and_network
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] post_xml_callback=post_xml_callback)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4978, in _create_domain
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] guest.launch(pause=pause)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 145, in launch
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self._encoded_xml, errors='ignore')
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] self.force_reraise()
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(self.type_, self.value, self.tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 140, in launch
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] return self._domain.createWithFlags(flags)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] rv = execute(f, *args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] six.reraise(c, e, tb)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] rv = meth(*args, **kwargs)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] File "/var/lib/kolla/venv/lib/python2.7/site-packages/libvirt.py", line 1065, in createWithFlags
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03] libvirtError: internal error: Unable to add port tapb5ee7be4-49 to OVS bridge br-int
2017-02-01 23:33:05.184 7 ERROR nova.compute.manager [instance: 4c80aaf5-7935-46da-97a6-a46dd77eee03]

# libvirtd.log
2017-02-02 04:33:04.815+0000: 960: info : libvirt version: 2.0.0, package: 10.el7_3.4 (CentOS BuildSystem <http://bugs.centos.org>, 2017-01-17-23:37:48, c1bm.rdu2.centos.org)
2017-02-02 04:33:04.815+0000: 960: info : hostname: aio
2017-02-02 04:33:04.815+0000: 960: error : virExec:492 : Cannot find 'ovs-vsctl' in path: No such file or directory
2017-02-02 04:33:04.815+0000: 960: error : virNetDevOpenvswitchAddPort:155 : internal error: Unable to add port tapb5ee7be4-49 to OVS bridge br-int
2017-02-02 04:33:04.838+0000: 960: error : virExec:492 : Cannot find 'ovs-vsctl' in path: No such file or directory
2017-02-02 04:33:04.838+0000: 960: error : virNetDevOpenvswitchRemovePort:188 : internal error: Unable to delete port tapb5ee7be4-49 from OVS

after checked the libvirt source code, libvirtd container need openvswitch package [1]

[1] https://github.com/libvirt/libvirt/blob/master/src/util/virnetdevopenvswitch.c#L90

Jeffrey Zhang (jeffrey4l)
Changed in kolla:
assignee: nobody → Jeffrey Zhang (jeffrey4l)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/428001

Changed in kolla:
status: New → In Progress
Eduardo Gonzalez (egonzalez90)
Changed in kolla:
importance: Undecided → High
milestone: none → ocata-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/428001
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=3b963bb3ad396cef0d1fbf8f833717ea9f2556a6
Submitter: Jenkins
Branch: master

commit 3b963bb3ad396cef0d1fbf8f833717ea9f2556a6
Author: Jeffrey Zhang <email address hidden>
Date: Thu Feb 2 12:45:27 2017 +0800

    Add openvswitch package for nova-libvirt container

    Change-Id: I61ce3b695e66cd76f8f9a43863c94601d312e8fe
    Closes-Bug: #1661152

Changed in kolla:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 4.0.0.0rc1

This issue was fixed in the openstack/kolla 4.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.