Segmentation fault during garbage collection when using custom parser and resolver

Bug #1772660 reported by gcc on 2018-05-22
This bug affects 2 people
Affects Status Importance Assigned to Milestone

Bug Description

The attached script causes a segmentation fault during garbage collection, on Python3 ONLY, with this stack trace:

(gdb) bt
#0 __pyx_pf_4lxml_5etree_14_ParserContext_2__dealloc__ (__pyx_v_self=0x7fffed26db10) at src/lxml/etree.c:42031
#1 __pyx_pw_4lxml_5etree_14_ParserContext_3__dealloc__ (__pyx_v_self=0x7fffed26db10) at src/lxml/etree.c:41946
#2 __pyx_tp_dealloc_4lxml_5etree__ParserContext (o=0x7fffed26db10) at src/lxml/etree.c:21936
#3 0x00007ffff7a5d05a in delete_garbage (old=<optimized out>, collectable=<optimized out>) at Modules/gcmodule.c:868
#4 collect (generation=generation@entry=2, n_collected=n_collected@entry=0x0, n_uncollectable=n_uncollectable@entry=0x0, nofail=nofail@entry=1) at Modules/gcmodule.c:1019
#5 0x00007ffff7a5da81 in _PyGC_CollectNoFail () at Modules/gcmodule.c:1623
#6 0x00007ffff7a28a04 in PyImport_Cleanup () at Python/import.c:420
#7 0x00007ffff7a3af9a in Py_FinalizeEx () at Python/pylifecycle.c:612
#8 0x00007ffff7a5b9c9 in Py_Main (argc=argc@entry=2, argv=argv@entry=0x603010) at Modules/main.c:829
#9 0x0000000000400b1c in main (argc=2, argv=<optimized out>) at ./Programs/python.c:69

This does not happen with version 3.7.1, or on Python 2.

Python : sys.version_info(major=3, minor=6, micro=1, releaselevel='final', serial=0)
lxml.etree : (4, 2, 1, 0)
libxml used : (2, 9, 4)
libxml compiled : (2, 9, 4)
libxslt used : (1, 1, 29)
libxslt compiled : (1, 1, 29)

gcc (chris+ubuntu-qwirx) wrote :
Kelledin (kelledin-3) wrote :
Download full text (6.5 KiB)

I'm getting a similar segfault during the test_inplace rule, on lxml 2.4.3, python 3.6.6, on aarch64, compiled with gcc-8.2.0 on glibc-2.27.

   ... <snip>
   1611/1822s(x88.4%): txtt(validation)invalid_schema_namespace.(...atronTestCase)
   1822/1822e(100.0%):.txt (xpathxslt)
   Doctest: xpathxslt.txt
   Ran 1822 tests in 95.962s

   Comparing with ElementTree 1.3.0

       Python: sys.version_info(major=2, minor=7, micro=15, releaselevel='final', serial=0)
       lxml.etree: (4, 2, 3, 0)
       libxml used: (2, 9, 8)
       libxml compiled: (2, 9, 8)
       libxslt used: (1, 1, 32)
       libxslt compiled: (1, 1, 32)

   make[2]: *** [Makefile:71: test_inplace] Segmentation fault (core dumped)

Note that I pulled in the change from, because the coverage report code took about an hour and sucked up ~2GB of memory, only to spit out a spurious error about the first "cimport" statement it found. The same segfault happens with or without that change though.

My backtrace looks like this:

#0 0x0000007f8ac12350 in PyDict_GetItem () from /usr/lib/aarch64-linux-gnu/
No symbol table info available.
#1 0x0000007f8abfc24c in PyFrame_New () from /usr/lib/aarch64-linux-gnu/
No symbol table info available.
#2 0x0000007f894975d8 in __Pyx_TraceSetupAndCall (code=code@entry=0x7f897cf518 <__pyx_frame_code.70765>, frame=frame@entry=0x7ff0a77c28, tstate=0x1d7032f0, funcname=funcname@entry=0x7f89745f48 "__dealloc__",
    srcfile=srcfile@entry=0x7f8973fbd8 "src/lxml/parser.pxi", firstlineno=firstlineno@entry=530) at src/lxml/etree.c:257932
        type = <optimized out>
        value = <optimized out>
        traceback = <optimized out>
        retval = <optimized out>
#3 0x0000007f894ef5e8 in __pyx_pf_4lxml_5etree_14_ParserContext_2__dealloc__ (__pyx_v_self=0x7f848b1750) at src/lxml/etree.c:114106
        tstate = <optimized out>
        __pyx_frame = 0x0
        __pyx_t_2 = <optimized out>
        __Pyx_use_tracing = 0
        __pyx_t_1 = <optimized out>
        __pyx_t_3 = <optimized out>
        __pyx_frame_code = 0x7f84b3c4b0
        __pyx_frame = <optimized out>
        __Pyx_use_tracing = <optimized out>
        __pyx_t_1 = <optimized out>
        __pyx_t_2 = <optimized out>
        __pyx_t_3 = <optimized out>
        tstate = <optimized out>
        state = <optimized out>
        tstate = <optimized out>
        ret = <optimized out>
        tstate = <optimized out>
        state = <optimized out>
        tstate = <optimized out>
        ret = <o...


Kelledin (kelledin-3) wrote :

my mistake, that's apparently from Python 2.7.15 (I have both versions installed side-by-side).

Kelledin (kelledin-3) wrote :

Further elaboration: this does NOT occur on Python 3.6.6, only Python 2.7.15 (in my setup).

It also does not unless I build --with-cython. For reference I have Cython 0.28.4 installed on both my Python 2 and Python 3 build. If it matters, the Cython front-ends in /usr/bin are suffixed as "cython3", "cygdb3", etc. for the Python 3 variant (all front-ends are shebanged to run the correct python interpreter).

Kelledin (kelledin-3) wrote :

Scratch that last bit, happens on Python 3.6.6 as well. I have to leave Cython support off to get a stable build.

Kelledin (kelledin-3) wrote :

And...I'm wrong again, one last time (apparently I needed to slow down a bit). It is specifically the coverage that's breaking it, not cython. I can build --with-cython, but I can't run --coverage without getting a segfault at the end, on either version of python.

For reference my python-coverage module version is 4.5.1 (for both Python 2 and Python 3).

scoder (scoder) wrote :

Thanks for the reports. I'm pretty sure the OP was reporting a different problem. These seem entirely unrelated.

I cannot reproduce the OP's crash with the provided script. I'm using libxml2 2.9.4, Python 3.6.6 and the latest lxml, and I get an error message that looks correct, but no crash.

Regarding the second problem, I also cannot reproduce this, but I tried using latest master, latest Cython and gcc-7.3. Don't have gcc-8 at hand currently.

Changed in lxml:
status: New → Triaged
WHK (yan-uniko-102) wrote :

Same problem when using c api for python and call Py_FinalizeEx() using __del__() from python class.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers