OpenStack Compute (nova)

nova-compute fails with "AttributeError: 'NoneType' object has no attribute 'get'" after kilo upgrade

Bug #1460053 reported by Francois Deppierraz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
New
Undecided
Unassigned

Bug Description

On compute node 'compute2', nova-compute fails to start with the following exception:

2015-05-29 14:12:42.545 16355 ERROR nova.openstack.common.threadgroup [req-a1d0fd3b-e3ff-48af-a568-4198ca22e3bc - - - - -] 'NoneType' object has no attribute 'get'
Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 422, in _object_dispatch
    return getattr(target, method)(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 163, in wrapper
    result = fn(cls, context, *args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1152, in get_by_host_and_node
    expected_attrs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1068, in _make_instance_list
    expected_attrs=expected_attrs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 501, in _from_db_object
    db_inst.get('extra').get('numa_topology'))

AttributeError: 'NoneType' object has no attribute 'get'
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/threadgroup.py", line 145, in wait
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup x.wait()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/threadgroup.py", line 47, in wait
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/service.py", line 497, in run_service
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup service.start()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/service.py", line 183, in start
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1291, in pre_start_hook
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6240, in update_available_resource
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup rt.update_available_resource(context)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 402, in update_available_resource
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup self._update_available_resource(context, resources)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return f(*args, **kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 436, in _update_available_resource
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup 'numa_topology'])
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 161, in wrapper
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup args, kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 335, in object_class_action
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup objver=objver, args=args, kwargs=kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup retry=self.retry)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup timeout=timeout, retry=retry)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup retry=retry)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup raise result
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup AttributeError: 'NoneType' object has no attribute 'get'
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 422, in _object_dispatch
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup return getattr(target, method)(*args, **kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 163, in wrapper
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup result = fn(cls, context, *args, **kwargs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1152, in get_by_host_and_node
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup expected_attrs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1068, in _make_instance_list
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup expected_attrs=expected_attrs)
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 501, in _from_db_object
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup db_inst.get('extra').get('numa_topology'))
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup AttributeError: 'NoneType' object has no attribute 'get'
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup
2015-05-29 14:12:42.545 16355 TRACE nova.openstack.common.threadgroup

Both the compute node and the controller node are running the latest kilo ubuntu packages under Ubuntu 14.04.

root@compute2:~# dpkg -l nova-compute
ii nova-compute 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node base
root@compute2:~#

root@controller1:~# dpkg -l nova-conductor
ii nova-conductor 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - conductor service
root@controller1:~#

Markus Zoeller (markus_z) (mzoeller)
tags: added: unified-objects
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Already fixed either by:
https://review.openstack.org/#/c/169517/

or by:
https://review.openstack.org/#/c/183127/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.