| 2014-07-30 02:41:23 |
Angus Lees |
description |
mysqldb is a C library, and thus can't be monkeypatched by eventlet.
With eventlet, this series of events may lead to a deadlock:
1. greenlet A opens a mysql DB transaction, does some stuff
2. greenlet A yields, perhaps by making another unrelated blocking operation (eg: sending a network request)
3. greenlet B is scheduled and opens a mysql DB transaction that conflicts with A's
4. greenlet B enters mysqldb and the C call blocks, waiting until A completes
The python process is now wedged until a mysqldb deadlock timer expires and kills the process. Note that this is a deadlock on the client side, since A will never be scheduled again.
The mysql deadlock timer has a distinctive "Lock wait timeout exceeded; try restarting transaction" error message.
There are bugs relating to specific instances of this, like bug 1298355 and bug 1313794.
.. but fundamentally this issue is generic across any use of mysqldb + eventlet + any code that yields the greenlet while holding a lock inside the database server. This bug is to track the generic issue across all such instances of the above pattern. Note it is not specific to oslo.db.
See also this blueprint for a proposed fix (switch to a different mysql driver):
- https://blueprints.launchpad.net/oslo/+spec/switch-to-mysql-connector |
mysqldb is a C library, and thus can't be monkeypatched by eventlet.
With eventlet, this series of events may lead to a deadlock:
1. greenlet A opens a mysql DB transaction, does some stuff
2. greenlet A yields, perhaps by making another unrelated blocking operation (eg: sending a network request)
3. greenlet B is scheduled and opens a mysql DB transaction that conflicts with A's
4. greenlet B enters mysqldb and the C call blocks, waiting until A completes
Note that this is a deadlock on the client side, since A will never be scheduled again - and not a typical example of waiting for a transaction lock. The python process is now wedged until a mysqldb deadlock timer expires and B raises an exception.
The mysql deadlock timer has a distinctive "Lock wait timeout exceeded; try restarting transaction" error message.
There are bugs relating to specific instances of this, like bug 1298355 and bug 1313794.
.. but fundamentally this issue is generic across any use of mysqldb + eventlet + any code that yields the greenlet while holding a lock inside the database server. This bug is to track the generic issue across all such instances of the above pattern. Note it is not specific to oslo.db.
See also this blueprint for a proposed fix (switch to a different mysql driver):
- https://blueprints.launchpad.net/oslo/+spec/switch-to-mysql-connector |
|
