Pythoscope exits with maximum recursion depth exceeded exception

[Michal Kwiatkowski]

Found by Edward K. Ream (http://groups.google.com/group/pythoscope/msg/7a597ddba9e136b8).

When run on the Leo project (http://webpages.charter.net/edreamleo/front.html) trunk, Pythoscope raises maximum recursion depth exceeded exception.

$ pythoscope leo/core/leoNodes.py
INFO: Inspecting module launchLeo.py.
[big cut...]
INFO: Inspecting module leo\modes\latex.py.
ERROR: Oops, it seems internal Pythoscope error occured. Please file a bug report at https://bugs.launchpad.net/pythoscope

Traceback (most recent call last):
 File "c:\Python25\Scripts\pythoscope-script.py", line 8, in <module>
   load_entry_point('pythoscope==0.3.2', 'console_scripts', 'pythoscope')()
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\__init__.py",
line 176, in main
   generate_tests(args, force, template)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\__init__.py",
line 104, in generate_tests
   inspect_project(project)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\inspector\__init__.py",
line 11, in inspect_project
   updates = add_and_update_modules(project) +
add_and_update_points_of_entry(project)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\inspector\__init__.py",
line 36, in add_and_update_modules
   static.inspect_module(project, modpath)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\inspector\static.py",
line 119, in inspect_module
   return inspect_code(project, path, read_file_contents(path))
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\inspector\static.py",
line 126, in inspect_code
   visitor = descend(tree, ModuleVisitor)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\astvisitor.py",
line 58, in descend
   visitor.visit(tree)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\astvisitor.py",
line 234, in visit
   self.visit_node(tree)
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\pythoscope\astvisitor.py",
line 247, in visit_node
   if pattern.match(node, results):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 428, in match
   if not self._submatch(node, r):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 542, in _submatch
   for c, r in generate_matches(self.content, node.children):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 744, in generate_matches
   for c0, r0 in p.generate_matches(nodes):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 658, in generate_matches
   for count, r in self._recursive_matches(nodes, 0):

 [big snip]

 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 686, in _recursive_matches
   for c0, r0 in generate_matches(alt, nodes):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 744, in generate_matches
   for c0, r0 in p.generate_matches(nodes):
 File "c:\python25\lib\site-packages\pythoscope-0.3.2-py2.5.egg\lib2to3\pytree.py",
line 451, in generate_matches
   if nodes and self.match(nodes[0], r):
RuntimeError: maximum recursion depth exceeded

C:\leo.repo\trunk>

[Michal Kwiatkowski (ruby)]

Setting max recursion limit to 2500 (via sys.setrecursionlimit: http://docs.python.org/library/sys.html#sys.setrecursionlimit) in astvisitor.ASTVisitor.visit_node makes it work. Documentation page says that "the highest possible limit is platform-dependent" and doesn't mention what are the values for different platforms, so we should research that before making a change.

Still, that's only a workaround. Maybe there's a way to make lib2to3.pytree.WildcardPattern._recursive_matches iterative? That would fix the problem once and for all.

[Michal Kwiatkowski (ruby)]

This was a bug in lib2to3, filed on python tracker here: http://bugs.python.org/issue2532 . Fix was ported from Python svn r69516 into our trunk r256.

[Michal Kwiatkowski (ruby)]

Fix released in 0.4.1.