OpenStack Compute (nova)

libvirtError: XML error: Missing CPU model name lead to compute service fail to start

Bug #1447342 reported by jichenjc on 2015-04-22

This bug affects 3 people

	Status	Importance	Assigned to	Milestone
OpenStack Compute (nova)	Fix Released	High	Matt Riedemann
Kilo	Fix Released	High	Matt Riedemann	OpenStack Compute (nova) 2015.1.4
Liberty	Fix Released	High	Matt Riedemann

Bug Description

got following error and failed to start a compute service
not sure if we should disallow compute service to start
if 'libvirtError: XML error: Missing CPU model name' or not

2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/service.py", line 497, in run_service
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup service.start()
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/service.py", line 164, in start
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self.manager.init_host()
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 1258, in init_host
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self.driver.init_host(host=self.host)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 529, in init_host
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self._do_quality_warnings()
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 507, in _do_quality_warnings
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup caps = self._host.get_capabilities()
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/host.py", line 753, in get_capabilities
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup rv = execute(f, *args, **kwargs)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup six.reraise(c, e, tb)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup rv = meth(*args, **kwargs)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 3153, in baselineCPU
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup libvirtError: XML error: Missing CPU model name

Tags:

jichenjc (jichenjc) on 2015-04-22

Changed in nova:
assignee:	nobody → jichenjc (jichenjc)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-04-24: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/177151

Changed in nova:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-06-09: Change abandoned on nova (master)

Change abandoned by jichenjc (<email address hidden>) on branch: master
Review: https://review.openstack.org/177151
Reason: agree it's a case that we can make compute service not up

Revision history for this message

John Garbutt (johngarbutt) wrote on 2015-08-05:

patch is abandoned, saying this is a valid failure as things are misconfigured.

tags:	added: libvirt
Changed in nova:
status:	In Progress → Invalid

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-12-01:

Change abandoned by jichenjc (<email address hidden>) on branch: master
Review: https://review.openstack.org/177151

Revision history for this message

Matt Riedemann (mriedem) wrote on 2016-03-01:

Download full text (4.1 KiB)

Sam-I-Am was reporting this in the nova IRC channel, same failure but with more details:

http://paste.openstack.org/show/488781/

In his case the host capabilities xml does not have a cpu model, so setting cpu_mode=none in nova.conf doesn't help, the details are in the stack trace:

2016-03-01 16:02:55.830 22615 ERROR oslo_service.service [req-5b0007bc-76b1-4ce1-86bb-2a79f73db1e7 - - - - -] Error starting thread.
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service Traceback (most recent call last):
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_service/service.py", line 680, in run_service
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service service.start()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/service.py", line 183, in start
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service self.manager.init_host()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1309, in init_host
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service self.driver.init_host(host=self.host)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 643, in init_host
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service self._do_quality_warnings()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 621, in _do_quality_warnings
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service caps = self._host.get_capabilities()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 787, in get_capabilities
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service result = proxy_call(self._autowrap, f, *args, **kwargs)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service rv = execute(f, *args, **kwargs)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service six.reraise(c, e, tb)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2016-03-01...

Sam-I-Am was reporting this in the nova IRC channel, same failure but with more details:

http://paste.openstack.org/show/488781/

In his case the host capabilities xml does not have a cpu model, so setting cpu_mode=none in nova.conf doesn't help, the details are in the stack trace:

2016-03-01 16:02:55.830 22615 ERROR oslo_service.service [req-5b0007bc-76b1-4ce1-86bb-2a79f73db1e7 - - - - -] Error starting thread.
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service Traceback (most recent call last):
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_service/service.py", line 680, in run_service
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     service.start()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/service.py", line 183, in start
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     self.manager.init_host()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1309, in init_host
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     self.driver.init_host(host=self.host)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 643, in init_host
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     self._do_quality_warnings()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 621, in _do_quality_warnings
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     caps = self._host.get_capabilities()
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 787, in get_capabilities
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     result = proxy_call(self._autowrap, f, *args, **kwargs)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     rv = execute(f, *args, **kwargs)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     six.reraise(c, e, tb)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     rv = meth(*args, **kwargs)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/libvirt.py", line 3153, in baselineCPU
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service     if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service libvirtError: XML error: Missing CPU model name
2016-03-01 16:02:55.830 22615 ERROR oslo_service.service

danpb said this about the failure:

danpb	Sam-I-Am: oh that's totally bogus and unrelated to the cpu_mode/cpu_model config options	16:05
danpb	Sam-I-Am: in the host.py  get_capabilities() method before it does                           [self._caps.host.cpu.to_xml()],	16:05
danpb	Sam-I-Am: it should check whether self._cps.host.cpu has a non-None model	16:06
danpb	Sam-I-Am: if the model is None, it should skip the baselineCPU() api call

Changed in nova:
status:	Invalid → Confirmed
importance:	Undecided → High
assignee:	jichenjc (jichenjc) → nobody

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-01: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/286868

Changed in nova:
assignee:	nobody → Matt Riedemann (mriedem)
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-04: Fix merged to nova (master)

Reviewed: https://review.openstack.org/286868
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=04bbf658e7998d40ffbdb2b6467dfab8dc5fde55
Submitter: Jenkins
Branch: master

commit 04bbf658e7998d40ffbdb2b6467dfab8dc5fde55
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 1 17:36:38 2016 -0500

libvirt: don't attempt to get baseline cpu features if host cpu model is None

In certain cases, libvirt can't determine the host's CPU model.

This is fine when you're setting virt_type=qemu and cpu_mode=none,
for example (like with nested virtualization).

    If we can't determine the host's cpu model, don't attempt to get
    cpu features on startup of the compute service (since it will
    crash the service).

Change-Id: I81ae5a04c7b4eb84e976902a575d890d4e850151
Closes-Bug: #1447342

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

Thierry Carrez (ttx) wrote on 2016-03-16: Fix included in openstack/nova 13.0.0.0rc1

This issue was fixed in the openstack/nova 13.0.0.0rc1 release candidate.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-21: Fix proposed to nova (stable/liberty)

#10

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/295478

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-21: Fix proposed to nova (stable/kilo)

#11

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/295509

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-24: Fix merged to nova (stable/liberty)

#12

Reviewed: https://review.openstack.org/295478
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1a0e8fce204fa99290de4d6f886371c80ecd2b48
Submitter: Jenkins
Branch: stable/liberty

commit 1a0e8fce204fa99290de4d6f886371c80ecd2b48
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 1 17:36:38 2016 -0500

libvirt: don't attempt to get baseline cpu features if host cpu model is None

In certain cases, libvirt can't determine the host's CPU model.

This is fine when you're setting virt_type=qemu and cpu_mode=none,
for example (like with nested virtualization).

    If we can't determine the host's cpu model, don't attempt to get
    cpu features on startup of the compute service (since it will
    crash the service).

    Change-Id: I81ae5a04c7b4eb84e976902a575d890d4e850151
    Closes-Bug: #1447342
    (cherry picked from commit 04bbf658e7998d40ffbdb2b6467dfab8dc5fde55)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-05: Fix merged to nova (stable/kilo)

#13

Reviewed: https://review.openstack.org/295509
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4951e9e550b86e4135e3312d4957799463e7336d
Submitter: Jenkins
Branch: stable/kilo

commit 4951e9e550b86e4135e3312d4957799463e7336d
Author: Matt Riedemann <email address hidden>
Date: Tue Mar 1 17:36:38 2016 -0500

libvirt: don't attempt to get baseline cpu features if host cpu model is None

In certain cases, libvirt can't determine the host's CPU model.

This is fine when you're setting virt_type=qemu and cpu_mode=none,
for example (like with nested virtualization).

    If we can't determine the host's cpu model, don't attempt to get
    cpu features on startup of the compute service (since it will
    crash the service).

    Change-Id: I81ae5a04c7b4eb84e976902a575d890d4e850151
    Closes-Bug: #1447342
    (cherry picked from commit 04bbf658e7998d40ffbdb2b6467dfab8dc5fde55)
    (cherry picked from commit 1a0e8fce204fa99290de4d6f886371c80ecd2b48)

Revision history for this message

Davanum Srinivas (DIMS) (dims-v) wrote on 2016-04-14: Fix included in openstack/nova 12.0.3

#14

This issue was fixed in the openstack/nova 12.0.3 release.

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2016-05-10: Fix included in openstack/nova 2015.1.4

#15

This issue was fixed in the openstack/nova 2015.1.4 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-10:

#16

This issue was fixed in the openstack/nova 2015.1.4 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.