neutron

DBDeadlock: DELETE FROM ipallocationpools WHERE ipallocationpools.id

Bug #1465310 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
New
Undecided
Unassigned

Bug Description

http://logs.openstack.org/43/154243/11/check/check-tempest-dsvm-neutron-full/d652e1e/logs/screen-q-svc.txt.gz?level=TRACE#_2015-06-11_16_22_49_538

2015-06-11 16:22:49.538 ERROR neutron.plugins.ml2.plugin [req-0c922e0d-c21b-4e2c-95e5-99238bf591a4 NetworksTestJSON-461887195 NetworksTestJSON-361140185] Exception auto-deleting subnet 920e9dbe-58bd-4d5f-ab09-9a4cc5358121
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin Traceback (most recent call last):
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 701, in _delete_subnets
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self.delete_subnet(context, subnet.id)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 131, in wrapper
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin return f(*args, **kwargs)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 919, in delete_subnet
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin break
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 490, in __exit__
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self.rollback()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin compat.reraise(exc_type, exc_value, exc_tb)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 487, in __exit__
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self.commit()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 392, in commit
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self._prepare_impl()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 372, in _prepare_impl
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self.session.flush()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2004, in flush
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self._flush(objects)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2122, in _flush
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin transaction.rollback(_capture_exception=True)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin compat.reraise(exc_type, exc_value, exc_tb)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2086, in _flush
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin flush_context.execute()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin rec.execute(self)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 561, in execute
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin uow
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 249, in delete_obj
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin cached_connections, mapper, table, delete)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 853, in _emit_delete_statements
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin c = connection.execute(statement, del_objects)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin return meth(self, multiparams, params)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin return connection._execute_clauseelement(self, multiparams, params)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin compiled_sql, distilled_params
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin context)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin util.raise_from_cause(newraise, exc_info)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin reraise(type(exception), exception, tb=exc_tb)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin context)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin cursor.execute(statement, parameters)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 134, in execute
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin result = self._query(query)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 282, in _query
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin conn.query(q)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 768, in query
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 929, in _read_query_result
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin result.read()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1125, in read
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin first_packet = self.connection._read_packet()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 893, in _read_packet
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin packet.check_error()
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 369, in check_error
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin err.raise_mysql_exception(self._data)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin _check_mysql_exception(errinfo)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 115, in _check_mysql_exception
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin raise InternalError(errno, errorvalue)
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin DBDeadlock: (pymysql.err.InternalError) (1213, u'Deadlock found when trying to get lock; try restarting transaction') [SQL: u'DELETE FROM ipallocationpools WHERE ipallocationpools.id = %s'] [parameters: (u'9f6b49c2-cfb3-4616-b968-64b104bb81a3',)]
2015-06-11 16:22:49.538 4580 ERROR neutron.plugins.ml2.plugin

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiREJEZWFkbG9ja1wiIEFORCBtZXNzYWdlOlwiREVMRVRFIEZST00gaXBhbGxvY2F0aW9ucG9vbHMgV0hFUkUgaXBhbGxvY2F0aW9ucG9vbHMuaWRcIiBBTkQgdGFnczpcInNjcmVlbi1xLXN2Yy50eHRcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQzNDM3ODU3MTg1MX0=

Matt Riedemann (mriedem)
summary: - DBDeadlock when deleting a subnet in the gate
+ DBDeadlock: DELETE FROM ipallocationpools WHERE ipallocationpools.id
Revision history for this message
Oleg Bondarev (obondarev) wrote :

Seems to be a duplicate of bug 1464612 which Kevin is working on

Revision history for this message
qingzhou (qingzhhu) wrote :
Download full text (8.0 KiB)

I found this bug in Juno which still using python-mysql

---

2016-03-30 03:24:12.045 980 ERROR neutron.api.v2.resource
[req-255ee495-9234-4eba-89ae-3ad4da1a9781 None] delete failed
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource Traceback (most
recent call last):
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 87, in
resource
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource result =
method(request=request, **args)
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 476, in
delete
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
obj_deleter(request.context, id, **kwargs)
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line
762, in delete_subnet
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource break
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
self.gen.next()
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/contextlib.py", line 121, in nested
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource if
exit(*exc):
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 470,
in __exit__
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
self.rollback()
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __exit__
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
compat.reraise(exc_type, exc_value, exc_tb)
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 467,
in __exit__
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource self.commit()
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 377,
in commit
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
self._prepare_impl()
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 357,
in _prepare_impl
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
self.session.flush()
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1919,
in flush
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
self._flush(objects)
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2037,
in _flush
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource
transaction.rollback(_capture_exception=True)
2016-03-30 03:24:12.045 980 TRACE neutron.api.v2.resource File
"/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __ex...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.