Deadlock found when trying to get lock; try restarting transaction (image_properties)

Bug #1344657 reported by Joshua Harlow on 2014-07-19
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Glance
Undecided
Unassigned

Bug Description

Still seeing deadlocks occur when updating image properties:

2014-07-19 04:10:55.484 20377 INFO glance.wsgi.server [858953c7-3f21-470e-af68-bfa35cfd38d1 62f09a8e53d44a8d841eabdce3d8f61e a25b4b81a3e54551bf168ef3a27d33aa - - -] Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 384, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token.py", line 559, in __call__
    return self._app(env, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/new/glance/glance/common/wsgi.py", line 381, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/new/glance/glance/common/wsgi.py", line 647, in __call__
    request, **action_args)
  File "/opt/stack/new/glance/glance/common/wsgi.py", line 671, in dispatch
    return method(*args, **kwargs)
  File "/opt/stack/new/glance/glance/common/utils.py", line 428, in wrapped
    return func(self, req, *args, **kwargs)
  File "/opt/stack/new/glance/glance/registry/api/v1/images.py", line 364, in delete
    deleted_image = self.db_api.image_destroy(req.context, id)
  File "/opt/stack/new/glance/glance/db/sqlalchemy/api.py", line 135, in image_destroy
    _image_property_delete_all(context, image_id, delete_time, session)
  File "/opt/stack/new/glance/glance/db/sqlalchemy/api.py", line 964, in _image_property_delete_all
    session)
  File "/opt/stack/new/glance/glance/db/sqlalchemy/api.py", line 925, in _image_child_entry_delete_all
    count = query.update({"deleted": True, "deleted_at": delete_time})
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2690, in update
    update_op.exec_()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 816, in exec_
    self._do_exec()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 913, in _do_exec
    update_stmt, params=self.query._params)
  File "/opt/stack/new/glance/glance/openstack/common/db/sqlalchemy/session.py", line 444, in _wrap
    _raise_if_deadlock_error(e, self.bind.dialect.name)
  File "/opt/stack/new/glance/glance/openstack/common/db/sqlalchemy/session.py", line 427, in _raise_if_deadlock_error
    raise exception.DBDeadlock(operational_error)
DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE image_properties SET updated_at=%s, deleted_at=%s, deleted=%s WHERE image_properties.image_id = %s AND image_properties.deleted = false' (datetime.datetime(2014, 7, 19, 4, 10, 55, 320775), datetime.datetime(2014, 7, 19, 4, 10, 55, 310447), 1, '98a1f0c1-fbd9-4cd0-83fc-bc2fc547b4ab')

http://logs.openstack.org/75/108175/2/check/check-tempest-dsvm-full/5f85a7e/logs/screen-g-reg.txt.gz#_2014-07-19_04_10_55_477

Joshua Harlow (harlowja) on 2014-07-19
summary: - eadlock found when trying to get lock; try restarting transaction
+ Deadlock found when trying to get lock; try restarting transaction
+ (image_properties)
Ankit Agrawal (ankitagrawal) wrote :

Hi Joshua,

Could you please provide steps to reproduce this bug.

Changed in glance:
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers