Conductor sqlalchemy session conflict

[Matthew Van Dijk]

The Trove SQLAlchemy handler caches the session. This should not be done for multi-threaded apps: http://docs.sqlalchemy.org/en/rel_1_0/orm/contextual.html

So far I have noticed this when bringing up a cluster. The conductor receives multiple heartbeats in a short span of time and the shared session causes an out of sync exception. The stack is below:

2015-08-04 20:03:05.161 ERROR oslo_messaging.rpc.dispatcher [-] Exception during message handling: (pymysql.err.OperationalError) (2014, 'Command Out of Sync')
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/trove/trove/conductor/manager.py", line 88, in heartbeat
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher instance_id=instance_id)
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/trove/trove/db/models.py", line 96, in find_by
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher model = cls.get_by(**conditions)
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/trove/trove/db/models.py", line 117, in get_by
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher return get_db_api().find_by(cls, **cls._process_conditions(kwargs))
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/trove/trove/db/sqlalchemy/api.py", line 50, in find_by
2015-08-04 20:03:05.161 TRACE oslo_messaging.rpc.dispatcher return _query_by(model, **kwargs).first()

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/209269

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matthew Van Dijk (<email address hidden>) on branch: master
Review: https://review.openstack.org/209269

[Matthew Van Dijk (mvandijk)]

Fixed by https://review.openstack.org/#/c/245980/