Ubuntu
yara-python package

Rule with three strings crashes with SIGSEGV

Bug #1877006 reported by James Fraser on 2020-05-06

This bug affects 3 people

Affects		Status	Importance	Assigned to	Milestone
	yara-python (Ubuntu)	Confirmed	Undecided	Unassigned

Bug Description

Ubuntu 20.04 (Focal)
libyara3:amd64 3.9.0-1
python3-yara 3.11.0-1build1
python3.8 3.8.2-1ubuntu1.1

The following is a distilled python program that will crash every time with a segmentation fault:

##############################
import yara

rules = """
rule test
{
    strings:
        $ = "asdf"
        $ = "jkl"
        $ = "add_action"
    condition:
        any of them
}
"""

r = yara.compile(source=rules)
input_path='/etc/issue.net'
r.match(input_path)
##############################

Removing one of the strings is enough to get it to execute correctly.

If I remove the apt package and install yara-python from pip (including the same version, 3.11.0) it does not crash and executes correctly.

I don't know how helpful it is but here's a backtrace with python3.8-dbg installed, I can't find any debug symbols for python3-yara:

#0 0x00007ffff74e2ab7 in yara_callback () from /usr/lib/python3/dist-packages/yara.cpython-38-x86_64-linux-gnu.so
#1 0x00007ffff74c36e3 in yr_scanner_scan_mem_blocks () from /lib/x86_64-linux-gnu/libyara.so.3
#2 0x00007ffff74c12ae in yr_rules_scan_mem_blocks () from /lib/x86_64-linux-gnu/libyara.so.3
#3 0x00007ffff74c135c in yr_rules_scan_mem () from /lib/x86_64-linux-gnu/libyara.so.3
#4 0x00007ffff74c1401 in yr_rules_scan_file () from /lib/x86_64-linux-gnu/libyara.so.3
#5 0x00007ffff74e42d7 in ?? () from /usr/lib/python3/dist-packages/yara.cpython-38-x86_64-linux-gnu.so
#6 0x00000000005f1625 in cfunction_call_varargs (kwargs=<optimized out>, args=<optimized out>, func=<built-in method match of yara.Rules object at remote 0x7ffff75a2a50>)
    at ../Objects/call.c:772
#7 PyCFunction_Call (func=<built-in method match of yara.Rules object at remote 0x7ffff75a2a50>, args=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:772
#8 0x00000000005f2246 in _PyObject_MakeTpCall (callable=<built-in method match of yara.Rules object at remote 0x7ffff75a2a50>, args=<optimized out>, nargs=<optimized out>,
    keywords=<optimized out>) at ../Include/internal/pycore_pyerrors.h:13
#9 0x000000000056c70f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffff75917b8,
    callable=<built-in method match of yara.Rules object at remote 0x7ffff75a2a50>) at ../Include/cpython/abstract.h:125
#10 _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffff75917b8, callable=<built-in method match of yara.Rules object at remote 0x7ffff75a2a50>)
    at ../Include/cpython/abstract.h:115
#11 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x95aec0) at ../Python/ceval.c:4987
#12 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3469
#13 0x00000000005654d2 in PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7ffff7591640, for file test.py, line 17, in <module> ()) at ../Python/ceval.c:741
#14 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>,
    kwargs=0x0, kwcount=<optimized out>, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at ../Python/ceval.c:4298
#15 0x0000000000686d53 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=<optimized out>,
    globals=<optimized out>, _co=<optimized out>) at ../Python/ceval.c:4327
#16 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:718
#17 0x0000000000676101 in run_eval_code_obj (co=0x7ffff74fc240,
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'},
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'}) at ../Python/pythonrun.c:1125
#18 0x000000000067617f in run_mod (mod=<optimized out>, filename=<optimized out>,
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'},
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'}, flags=<optimized out>, arena=<optimized out>) at ../Python/pythonrun.c:1147
#19 0x0000000000676237 in PyRun_FileExFlags (fp=0x959b30, filename_str=<optimized out>, start=<optimized out>,
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'},
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='test.py') at remote 0x7ffff75dffa0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761d0e0>, '__file__': 'test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f2950>, 'rules': '\nrule test\n{\n strings:\n $ = "asdf"\n $ = "jkl"\n $ = "add_action"\n condition:\n any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a2a50>, 'input_path': '/etc/issue.net'}, closeit=1, flags=0x7fffffffe348) at ../Python/pythonrun.c:1063
#20 0x00000000006782ba in PyRun_SimpleFileExFlags (fp=0x959b30, filename=<optimized out>, closeit=1, flags=0x7fffffffe348) at ../Python/pythonrun.c:428
#21 0x00000000006af5ce in pymain_run_file (cf=0x7fffffffe348, config=0x95a2b0) at ../Modules/main.c:381
#22 pymain_run_python (exitcode=0x7fffffffe340) at ../Modules/main.c:565
#23 Py_RunMain () at ../Modules/main.c:644
#24 0x00000000006af959 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:698
#25 0x00007ffff7df80b3 in __libc_start_main (main=0x4ec640 <main>, argc=2, argv=0x7fffffffe528, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7fffffffe518) at ../csu/libc-start.c:308
#26 0x00000000005f69be in _start () at ../Objects/obmalloc.c:1233

Let me know if you require any further information. Thanks!

Revision history for this message

James Fraser (fwaggle) wrote on 2020-05-14:

Download full text (9.4 KiB)

I managed to track down the package with the debug symbols for python3-yara, I'm not sure if it's super helpful as my gdb skills leave a lot to be desired:

I managed to track down the package with the debug symbols for python3-yara, I'm not sure if it's super helpful as my gdb skills leave a lot to be desired:

Program received signal SIGSEGV, Segmentation fault.
yara_callback (message=2, message_data=0x9fbb20, user_data=<optimized out>) at yara-python.c:729
warning: Source file is more recent than executable.
729	      object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
(gdb) p m
$1 = (YR_MATCH *) 0xffffffffffffffff
(gdb) bt
#0  yara_callback (message=2, message_data=0x9fbb20, user_data=<optimized out>) at yara-python.c:729
#1  0x00007ffff74c46e3 in yr_scanner_scan_mem_blocks () from /lib/x86_64-linux-gnu/libyara.so.3
#2  0x00007ffff74c22ae in yr_rules_scan_mem_blocks () from /lib/x86_64-linux-gnu/libyara.so.3
#3  0x00007ffff74c235c in yr_rules_scan_mem () from /lib/x86_64-linux-gnu/libyara.so.3
#4  0x00007ffff74c2401 in yr_rules_scan_file () from /lib/x86_64-linux-gnu/libyara.so.3
#5  0x00007ffff74e52d7 in Rules_match (self=<yara.Rules at remote 0x7ffff75a3960>, args=<optimized out>, keywords=<optimized out>) at yara-python.c:1459
#6  0x00000000005f1625 in cfunction_call_varargs (kwargs=<optimized out>, args=<optimized out>, 
    func=<built-in method match of yara.Rules object at remote 0x7ffff75a3960>) at ../Objects/call.c:772
#7  PyCFunction_Call (func=<built-in method match of yara.Rules object at remote 0x7ffff75a3960>, args=<optimized out>, kwargs=<optimized out>)
    at ../Objects/call.c:772
#8  0x00000000005f2246 in _PyObject_MakeTpCall (callable=<built-in method match of yara.Rules object at remote 0x7ffff75a3960>, args=<optimized out>, 
    nargs=<optimized out>, keywords=<optimized out>) at ../Include/internal/pycore_pyerrors.h:13
#9  0x000000000056c70f in _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffff75927c0, 
    callable=<built-in method match of yara.Rules object at remote 0x7ffff75a3960>) at ../Include/cpython/abstract.h:125
#10 _PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffff75927c0, 
    callable=<built-in method match of yara.Rules object at remote 0x7ffff75a3960>) at ../Include/cpython/abstract.h:115
#11 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x95aec0) at ../Python/ceval.c:4987
#12 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3469
#13 0x00000000005654d2 in PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7ffff7592640, for file ./test.py, line 18, in <module> ()) at ../Python/ceval.c:741
#14 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kwnames=<optimized out>, kwargs=0x0, kwcount=<optimized out>, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0)
    at ../Python/ceval.c:4298
#15 0x0000000000686d53 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=<optimized out>, 
    globals=<optimized out>, _co=<optimized out>) at ../Python/ceval.c:4327
#16 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:718
#17 0x0000000000676101 in run_eval_code_obj (co=0x7ffff74fd240, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}, 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}) at ../Python/pythonrun.c:1125
#18 0x000000000067617f in run_mod (mod=<optimized out>, filename=<optimized out>, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0--Type <RET> for more, q to quit, c to continue without paging--
eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}, 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}, flags=<optimized out>, arena=<optimized out>) at ../Python/pythonrun.c:1147
#19 0x0000000000676237 in PyRun_FileExFlags (fp=0x959b30, filename_str=<optimized out>, start=<optimized out>, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}, 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='./test.py') at remote 0x7ffff75e0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff761e0e0>, '__file__': './test.py', '__cached__': None, 'yara': <module at remote 0x7ffff74f4680>, 'rules': '\nrule test\n{\n    strings:\n        $ = "asdf"\n        $ = "jkl"\n        $ = "add_action"\n    condition:\n        any of them\n}\n', 'r': <yara.Rules at remote 0x7ffff75a3960>, 'input_path': '/etc/issue.net'}, closeit=1, flags=0x7fffffffe3f8) at ../Python/pythonrun.c:1063
#20 0x00000000006782ba in PyRun_SimpleFileExFlags (fp=0x959b30, filename=<optimized out>, closeit=1, flags=0x7fffffffe3f8) at ../Python/pythonrun.c:428
#21 0x00000000006af5ce in pymain_run_file (cf=0x7fffffffe3f8, config=0x95a2b0) at ../Modules/main.c:381
#22 pymain_run_python (exitcode=0x7fffffffe3f0) at ../Modules/main.c:565
#23 Py_RunMain () at ../Modules/main.c:644
#24 0x00000000006af959 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:698
#25 0x00007ffff7df90b3 in __libc_start_main (main=0x4ec640 <main>, argc=2, argv=0x7fffffffe5d8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe5c8) at ../csu/libc-start.c:308
#26 0x00000000005f69be in _start () at ../Objects/obmalloc.c:1233

Interestingly, if I `apt-get source python3-yara` and build it myself, then install my self-built package, it does not crash with the same script input.

If I install the apt version again (after clearing the apt cache and removing the self-built package), I can get it to crash again. If I modify the rules in test script slightly to force a match, then match is walked correctly, before the foreach runs again with garbage data:

rules = """
rule test
{
    strings:
        $ = "Ubuntu"
        $ = "asdf"
        $ = "jkl"
    condition:
        any of them
}
"""

(gdb) break yara-python.c:729
No source file named yara-python.c.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (yara-python.c:729) pending.
(gdb) run
Starting program: /usr/bin/python3 ./test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, yara_callback (message=1, message_data=0x9fbb20, user_data=<optimized out>) at yara-python.c:729
warning: Source file is more recent than executable.
729	      object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
(gdb) p m->data
$1 = (const uint8_t *) 0x7ffff703a04f "Ubuntu"
(gdb) c
Continuing.

Breakpoint 1, yara_callback (message=1, message_data=0x9fbb20, user_data=<optimized out>) at yara-python.c:729
729	      object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
(gdb) p m->data
Cannot access memory at address 0x17
(gdb)

Comparing the build log for the package to mine there are a few package versions, so maybe this is something that's already fixed if a package is rebuilt with newer libs, but I'm about out of my depth. :(

Revision history for this message

Launchpad Janitor (janitor) wrote on 2020-06-18:

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in yara-python (Ubuntu):
status:	New → Confirmed

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntuyara-python package

Rule with three strings crashes with SIGSEGV

Bug Description

Other bug subscribers

Remote bug watches

Ubuntu
yara-python package