"maximum recursion depth exceeded" when having more than 500 terms in search field

[Adam Ryczkowski]

...and if little less then 500, it takes ages to process.

The bug reads like this:
calibre, version 1.17.0
ERROR: Unhandled exception: <b>RuntimeError</b>:maximum recursion depth exceeded

calibre 1.17 isfrozen: True is64bit: True
Linux-3.8.0-34-generic-x86_64-with-debian-wheezy-sid Linux ('64bit', 'ELF')
('Linux', '3.8.0-34-generic', '#49-Ubuntu SMP Tue Nov 12 18:00:10 UTC 2013')
Python 2.7.5
Linux: ('debian', 'wheezy/sid', '')
Traceback (most recent call last):
  File "site-packages/calibre/gui2/library/views.py", line 1024, in search_proxy
  File "site-packages/calibre/gui2/library/models.py", line 424, in search
  File "site-packages/calibre/db/view.py", line 269, in search
  File "site-packages/calibre/db/view.py", line 304, in search_getting_ids
  File "site-packages/calibre/db/cache.py", line 57, in call_func_with_lock
  File "site-packages/calibre/db/cache.py", line 889, in search
  File "site-packages/calibre/db/search.py", line 842, in __call__
  File "site-packages/calibre/db/search.py", line 879, in _do_search
  File "site-packages/calibre/db/search.py", line 477, in parse
  File "site-packages/calibre/utils/search_query_parser.py", line 327, in parse
  File "site-packages/calibre/utils/search_query_parser.py", line 348, in _parse
  File "site-packages/calibre/utils/search_query_parser.py", line 356, in evaluate
  File "site-packages/calibre/utils/search_query_parser.py", line 370, in evaluate_or
<the last two lines are repeated 473 times>
  File "site-packages/calibre/utils/search_query_parser.py", line 369, in evaluate_or
  File "site-packages/calibre/utils/search_query_parser.py", line 356, in evaluate
  File "site-packages/calibre/utils/search_query_parser.py", line 399, in evaluate_token
  File "site-packages/calibre/utils/search_query_parser.py", line 403, in _get_matches
  File "site-packages/calibre/db/search.py", line 499, in get_matches
  File "site-packages/calibre/db/search.py", line 457, in field_metadata
RuntimeError: maximum recursion depth exceeded

To reproduce, it is enough to make a query with length >500 terms like that (I use custom numerical column cid):
#cid:=1953 or =1863 or =1492 or =1484 or =1472 or =2339 or =2033 or =2301 or =1943 or =2150 or =2323 or =2235 or =2388 or ...

Such query are produced by the http://www.mobileread.com/forums/showthread.php?t=211137&highlight=Recoll+Full+Text+Search+Plugin plug-in.

I'd like to know, if fixing this problem is feasible (I understand, that normal user would not post a query of such length), and if not, what would be the best strategy to re-write the Recoll Plugin.

I never programmed anything in Python yet, but necessity is a mother of acquiring new skills :-).

[Kovid Goyal (kovid)]

Trying to optimize the search engine for such artificially generated queries is way too much work. If you need to do this kind fo search, I suggest the following approach:

1) Do the search your self to get the set of matched book ids in the plugin
2) Use the mark books feature to mark the matched books (call set_marked_ids())
3) Then simply run the search #marked:true

This will give you excellent performance.