lazy load operation of attribute 'system_metadata' cannot proceed

Bug #1117232 reported by Wenhao Xu
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Wenhao Xu

Bug Description

branch: master
commit: ee955d272ab4eef2e8371f2e460454058ce7e46c

This happens sometimes when I restart nova-conductor and nova-compute services. The notification via rabbitmq is enabled in nova.
Is this the same cause of bug https://bugs.launchpad.net/nova/+bug/785816?

nova-conductor trace is here:

2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp
2013-02-06 19:21:20,250.250 ERROR nova.openstack.common.rpc.amqp [req-6c4474a3-ea95-4b40-9edb-a131c9a32aee None None] Exception during message handling
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/openstack/common/rpc/amqp.py", line 276, in _process_data
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/conductor/manager.py", line 93, in instance_get_all_by_host
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp return jsonutils.to_primitive(result)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/openstack/common/jsonutils.py", line 97, in to_primitive
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp level=level))
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/openstack/common/jsonutils.py", line 108, in to_primitive
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp return to_primitive(dict(value.iteritems()),
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/db/sqlalchemy/models.py", line 102, in iteritems
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp local = dict(self)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/home/wenhao/workspace/openstack/nova/nova/db/sqlalchemy/models.py", line 91, in next
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp return n, getattr(self, n)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 168, in __get__
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp return self.impl.get(instance_state(instance),dict_)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 453, in get
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp value = self.callable_(state, passive)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 481, in _load_for_state
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp (mapperutil.state_str(state), self.key)
2013-02-06 19:21:20,250.250 31860 TRACE nova.openstack.common.rpc.amqp DetachedInstanceError: Parent instance <Instance at 0x443ba10> is not bound to a Session; lazy load operation of attribute 'system_metadata' cannot proceed

Wenhao Xu (wenhao-x)
description: updated
Wenhao Xu (wenhao-x)
Changed in nova:
assignee: nobody → Wenhao Xu (wenhao-x)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/21298

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/21298
Committed: http://github.com/openstack/nova/commit/ecb1f61412cbeb074ebd8ba5a123e2a55821c148
Submitter: Jenkins
Branch: master

commit ecb1f61412cbeb074ebd8ba5a123e2a55821c148
Author: Wenhao Xu <email address hidden>
Date: Wed Feb 6 19:49:38 2013 +0800

    Fix lazy load 'system_metadata' failed problem.

    When using lazy load of SQLAlchemy, parent instance may have
    a problem not bouding to a session object and fail the lazy loading
    system_metadata.
    Using joinedload to eagly load the system_metadata in
    _instance_get_all_query to fix the problem.

    Fixes: bug 1117232
    Change-Id: Id24ca534f6eb57167aee22aca12c1274c90baf3c

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-3 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.