active etree method in thread crashes python process on shutdown
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
lxml |
Triaged
|
Low
|
Unassigned |
Bug Description
Encountered on:
Python : sys.version_
lxml.etree : (3, 3, 3, 0)
libxml used : (2, 9, 1)
libxml compiled : (2, 9, 1)
libxslt used : (1, 1, 28)
libxslt compiled : (1, 1, 28)
And reproduced on:
Python : sys.version_
lxml.etree : (3, 3, 5, 0)
libxml used : (2, 9, 1)
libxml compiled : (2, 9, 1)
libxslt used : (1, 1, 28)
libxslt compiled : (1, 1, 28)
When a thread is running etree.fromstring while the main process is trying to finalize the interpreter on shutdown, the fromstring call might cause a segfault.
Following is a minimal reproduction case. It segfaults every ~15 runs on my machine:
from lxml import etree
import threading
class Thread(
def __init__(self):
self.daemon = True
def run(self):
while True:
Thread().start()
>
Sample stacktrace:
Core was generated by `/usr/bin/python -B -m <program>'.
Program terminated with signal 11, Segmentation fault.
#0 0x000001fff6cfc488 in ?? () from /lib/python2.
$1 = -99
XXX Signaled thread
#0 0x000001fff6cfc488 in ?? () from /lib/python2.
#1 0x000001fff6cfca40 in ?? () from /lib/python2.
#2 0x000001fff6e86d90 in ?? () from /lib/python2.
#3 0x000001fff7cb40b8 in PyCFunction_Call () from /lib/libpython2
#4 0x000001fff7d65220 in PyEval_EvalFrameEx () from /lib/libpython2
#5 0x000001fff7d67128 in PyEval_EvalFrameEx () from /lib/libpython2
#6 0x000001fff7d67128 in PyEval_EvalFrameEx () from /lib/libpython2
#7 0x000001fff7d68368 in PyEval_EvalCodeEx () from /lib/libpython2
#8 0x000001fff7c89f70 in ?? () from /lib/libpython2
#9 0x000001fff7c38d18 in PyObject_Call () from /lib/libpython2
#10 0x000001fff7c5ec00 in ?? () from /lib/libpython2
#11 0x000001fff7c38d18 in PyObject_Call () from /lib/libpython2
#12 0x000001fff7d5b068 in PyEval_
#13 0x000001fff7db6d28 in ?? () from /lib/libpython2
#14 0x000001fff7ba8f48 in ?? () from /lib/libpthread
#15 0x000001fff7922740 in clone () from /lib/libc.so.6
$2 = -99
XXX All Threads
Thread 2 (LWP 942):
#0 0x000001fff7bb3548 in sem_wait () from /lib/libpthread
#1 0x000001fff7dae618 in PyThread_
#2 0x000001fff7d68698 in PyEval_
#3 0x000001fff7c764f0 in ?? () from /lib/libpython2
#4 0x000001fff7c79ea8 in ?? () from /lib/libpython2
#5 0x000001fff7cae800 in ?? () from /lib/libpython2
#6 0x000001fff7cae800 in ?? () from /lib/libpython2
#7 0x000001fff7d81388 in _PyImport_Fini () from /lib/libpython2
#8 0x000001fff7d98e48 in Py_Finalize () from /lib/libpython2
#9 0x000001fff7d99118 in Py_Exit () from /lib/libpython2
#10 0x000001fff7d99408 in ?? () from /lib/libpython2
#11 0x000001fff7d99858 in PyErr_PrintEx () from /lib/libpython2
#12 0x000001fff7db0e78 in ?? () from /lib/libpython2
#13 0x000001fff7db1f48 in Py_Main () from /lib/libpython2
#14 0x000001fff77ee4c8 in __libc_start_main () from /lib/libc.so.6
#15 0x00000000010007b0 in _start ()
I'd consider patches that fix this, but I don't really consider it important. Doing actual processing in daemon threads sounds like a bad design in the first place.