DetachedInstanceError on trying to access share['share_server']

Bug #1700660 reported by Goutham Pacha Ravi on 2017-06-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Manila
Undecided
NidhiMittalHada

Bug Description

The share_server attributes of a share instance are currently lazy loaded. Our default sqlalchemy sessionmaker behavior is to create sessions with "expire_on_commit" set to False.

If expire_on_commit is set to True, per documentation here: http://docs.sqlalchemy.org/en/latest/orm/session_basics.html, "all instances will be fully expired after each commit(), so that all attribute/object access subsequent to a completed transaction will load from the most recent database state."

Lazy loading is efficient for fields that are not used in most queries; however, whether attributes are loaded immediately, or lazy loaded, they are available for consumption when handed off to share drivers. When an attribute hasn't been loaded, sqlalchemy expects to see "null" in place of the attribute.. the thing is, the attribute can indeed be a python None.

For example, I tried using the "share_server" attribute of a share instance in my driver and it blew up like this:

Jun 25 06:47:10.872813 openstack8 manila-share[8112]:ERROR manila.share.manager [req-b40f3ef3-ff6c-4c14-bc7f-46bf2069c98f e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956cc5d2
3bbe937a] Share instance a88eda1e-931d-49f1-8fc1-545cf453b65c failed on creation.: DetachedInstanceError: Parent instance <ShareInstance at 0x7f19013998d0> is not bound to a Session; lazy load operation o
f attribute 'share_server' cannot proceed
Jun 25 06:47:10.873518 openstack8 manila-share[8112]:WARNING manila.share.manager [req-b40f3ef3-ff6c-4c14-bc7f-46bf2069c98f e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956cc5
d23bbe937a] Share instance information in exception can not be written to db because it contains {} and it is not a dictionary.: DetachedInstanceError: Parent instance <ShareInstance at 0x7f19013998d0> is
 not bound to a Session; lazy load operation of attribute 'share_server' cannot proceed
Jun 25 06:47:11.567449 openstack8 manila-share[8112]:ERROR oslo_messaging.rpc.server [req-b40f3ef3-ff6c-4c14-bc7f-46bf2069c98f e83153b7fe7a4886a9e45ee4ae9de822 57461a16b8324c3d956c
c5d23bbe937a] Exception during message handling: DetachedInstanceError: Parent instance <ShareInstance at 0x7f19013998d0> is not bound to a Session; lazy load operation of attribute 'share_server' cannot
proceed
Jun 25 06:47:11.567663 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server Traceback (most recent call last):
Jun 25 06:47:11.567833 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 153, in _pro
cess_incoming
Jun 25 06:47:11.568064 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Jun 25 06:47:11.568403 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in
dispatch
Jun 25 06:47:11.568725 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Jun 25 06:47:11.569031 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in
_do_dispatch
Jun 25 06:47:11.569359 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
Jun 25 06:47:11.569686 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 169, in wrapped
Jun 25 06:47:11.570014 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return f(self, *args, **kwargs)
Jun 25 06:47:11.570300 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/utils.py", line 552, in wrapper
Jun 25 06:47:11.570602 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return func(self, *args, **kwargs)
Jun 25 06:47:11.570933 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 1639, in create_share_instance
Jun 25 06:47:11.571301 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server {'status': constants.STATUS_ERROR}
Jun 25 06:47:11.571535 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__

Jun 25 06:47:11.571835 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server self.force_reraise()
Jun 25 06:47:11.572117 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jun 25 06:47:11.572440 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
Jun 25 06:47:11.572598 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 1610, in create_share_instance
Jun 25 06:47:11.572868 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server context, share_instance, share_server=share_server)
Jun 25 06:47:11.573171 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/drv_single_svm.py", line 49, in create_share
Jun 25 06:47:11.573477 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return self.library.create_share(context, share, **kwargs)
Jun 25 06:47:11.573780 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 25 06:47:11.574079 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server result = f(self, *args, **kwargs)
Jun 25 06:47:11.574433 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 443, in create_share
Jun 25 06:47:11.574759 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server self._allocate_container(share, vserver_client)
Jun 25 06:47:11.574986 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 25 06:47:11.575480 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server result = f(self, *args, **kwargs)
Jun 25 06:47:11.575812 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 468, in _allocate_container
Jun 25 06:47:11.576117 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server share, replica=replica)
Jun 25 06:47:11.576277 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 25 06:47:11.576434 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server result = f(self, *args, **kwargs)
Jun 25 06:47:11.576589 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 665, in _get_provisioning_options_for_share
Jun 25 06:47:11.576764 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server qos_policy_group = self._create_qos_policy_group(share, qos_specs)
Jun 25 06:47:11.577333 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/utils.py", line 86, in trace_wrapper
Jun 25 06:47:11.577500 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server result = f(self, *args, **kwargs)
Jun 25 06:47:11.577661 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 645, in _create_qos_policy_group
Jun 25 06:47:11.577819 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server vserver, __ = self._get_vserver(share_server=share.get('share_server'))
Jun 25 06:47:11.577989 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/models.py", line 70, in get
Jun 25 06:47:11.578146 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return getattr(self, key, default)
Jun 25 06:47:11.578324 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
Jun 25 06:47:11.578484 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server return self.impl.get(instance_state(instance), dict_)
Jun 25 06:47:11.578646 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 584, in get
Jun 25 06:47:11.578810 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server value = self.callable_(state, passive)
Jun 25 06:47:11.578975 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 530, in _load_for_state
Jun 25 06:47:11.579136 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server (orm_util.state_str(state), self.key)
Jun 25 06:47:11.579333 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server DetachedInstanceError: Parent instance <ShareInstance at 0x7f19013998d0> is not bound to a Session; lazy load operation of attribute 'share_server' cannot proceed
Jun 25 06:47:11.579514 openstack8 manila-share[8112]:TRACE oslo_messaging.rpc.server

Goutham Pacha Ravi (gouthamr) wrote :

There seem to be three solutions to this problem:

* Set the share_server model to be loaded immediately with share instances
* Set expire_on_commit to True in the sessionmaker
* Explicitly expire the session once the values have been captured from the database

Also, this goes without saying that drivers don't need to be handed sqlalchemy objects because they
don't need to perform any further database operations by themselves.

Changed in manila:
assignee: nobody → NidhiMittalHada (nidhimittal19)
Jason Grosso (jgrosso) wrote :

Is anyone still working on this solution??

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers