| 2014-01-08 17:38:14 |
Kashyap Chamarthy |
bug |
|
|
added bug |
| 2014-01-08 18:06:59 |
Kashyap Chamarthy |
bug watch added |
|
https://bugzilla.redhat.com/show_bug.cgi?id=1049391 |
|
| 2014-01-10 21:48:34 |
Matt Riedemann |
tags |
|
libvirt |
|
| 2014-01-15 22:15:22 |
Kyle Mestery |
bug |
|
|
added subscriber Kyle Mestery |
| 2014-01-27 19:28:59 |
Davanum Srinivas (DIMS) |
tags |
libvirt |
libvirt libvirt1x |
|
| 2014-02-03 16:38:12 |
Kashyap Chamarthy |
nova: status |
New |
Incomplete |
|
| 2014-02-03 16:38:23 |
Kashyap Chamarthy |
nova: status |
Incomplete |
Fix Committed |
|
| 2014-02-03 16:44:30 |
Kashyap Chamarthy |
nova: status |
Fix Committed |
Invalid |
|
| 2014-02-13 19:58:40 |
Ryan Beisner |
bug |
|
|
added subscriber Ryan Beisner |
| 2014-07-29 14:49:07 |
Chris J Arges |
bug task added |
|
nova (Ubuntu) |
|
| 2014-07-29 14:51:31 |
Chris J Arges |
nominated for series |
|
Ubuntu Trusty |
|
| 2014-07-29 14:51:31 |
Chris J Arges |
bug task added |
|
nova (Ubuntu Trusty) |
|
| 2014-07-29 15:19:47 |
Rafael David Tinoco |
description |
Description of problem
---------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
[Impact]
* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).
[Test Case]
* To be provided.
[Regression Potential]
* Fix is running in a big production without any problem.
* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.
[Other Info]
For rbd image backend for libvirt, the root partition will use the image size not 'disk' size. It lack of resize root volume in the codes.
--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
|
| 2014-07-29 15:22:43 |
Rafael David Tinoco |
description |
[Impact]
* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).
[Test Case]
* To be provided.
[Regression Potential]
* Fix is running in a big production without any problem.
* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.
[Other Info]
For rbd image backend for libvirt, the root partition will use the image size not 'disk' size. It lack of resize root volume in the codes.
--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
[Impact]
* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).
[Test Case]
* To be provided.
[Regression Potential]
* Fix is running in a big production without any problem.
* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.
[Other Info]
--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
|
| 2014-07-29 15:50:17 |
Rafael David Tinoco |
description |
[Impact]
* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).
[Test Case]
* To be provided.
[Regression Potential]
* Fix is running in a big production without any problem.
* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.
[Other Info]
--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
[Impact]
* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).
[Test Case]
* A two node openstack havana on ubuntu 14.04. Migrating a instance to other node.
- Migrate the instance
- Observe /var/log/nova/compute.log and /var/log/libvirt.log.
- /var/log/libvirtd.log: 2014-03-27 18:23:17.141+0000: 2659: error : virCPUDefParseXML:413 : XML error: CPU feature `wdt' specified more than once.
[Regression Potential]
* Same fix for 1303536 applied to Icehouse.
* Fix is running in a big production without any problem.
* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.
[Other Info]
--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------
Restarting openstack-nova-compute serivce fails with:
libvirtError: internal error: CPU feature `avx' specified more than once
Version
---------
$ rpm -q openstack-nova libvirt qemu-system-x86
openstack-nova-2014.1-0.4.b1.fc21.noarch
libvirt-1.1.3.2-1.fc20.x86_64
qemu-system-x86-1.6.1-3.fc20.x86_64
Test env
----------
A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)
How reproducible: Consistently.
Steps to Reproduce
------------------
$ systemctl restart openstack-nova-compute
Observe /var/log/nova/compute.log
Actual results
--------------
$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
Expected results
----------------
Compute service should start successfully.
Additional info
---------------
Status of openstack-nova-compute service
$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup
[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST
More contextual trace from compute.log:
[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .] |
|
| 2014-07-29 15:52:19 |
Chris J Arges |
nova (Ubuntu): status |
New |
Fix Released |
|
| 2014-07-29 16:09:55 |
Chris J Arges |
nova (Ubuntu Trusty): status |
New |
Fix Committed |
|
| 2014-07-29 16:09:58 |
Chris J Arges |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2014-07-29 16:10:02 |
Chris J Arges |
bug |
|
|
added subscriber SRU Verification |
| 2014-07-29 16:10:17 |
Chris J Arges |
tags |
libvirt libvirt1x |
libvirt libvirt1x verification-needed |
|
| 2014-08-06 17:50:33 |
Rafael David Tinoco |
tags |
libvirt libvirt1x verification-needed |
libvirt libvirt1x verification-done |
|
| 2014-08-06 19:22:11 |
Scott Kitterman |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2014-08-06 19:27:23 |
Launchpad Janitor |
nova (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2014-08-08 18:21:30 |
Launchpad Janitor |
branch linked |
|
lp:~corey.bryant/nova/2014.1.2 |
|
| 2014-08-11 14:16:47 |
Launchpad Janitor |
branch linked |
|
lp:~ubuntu-server-dev/nova/icehouse |
|
| 2016-01-26 05:06:58 |
刘志远 |
summary |
openstack-nova-compute service fails with - libvirtError: internal error: CPU feature `avx' specified more than once |
openstack-nova-compute service fails with - libvirtError: internal error: CPU feature `avx' specified more than once |
|
