Comment 2 for bug 1251353

Dan,

First, thanks for testing and I'm glad it generally improved things!

It sounds like Postgres decided to eat all the memory and then the linux OOM killer came along and shot down a backend, which would throw the db into recovery mode. If memory, I/O or CPU pressure were high enough, I could imagine this looking like an all-out crash.

That's working as intended, given the current code. The way we deal with bools is to create a nested tree of left joins and then apply the bool operator to that branch of the WHERE clause. This is a straight forward implementation that leverages postgres as much as possible, but past a certain number of joins, it's simply too much to handle.

I've been contemplating possible optimizations for a while. The way I would like to attack this problem is to teach the top-level QueryParser module to recognize situations where logically adjacent nodes use the same class and bool operator, and pull the contents of the down-link node into the parent, when the down-link is a leaf node of the parse tree. This would be a post-parsing optimization implemented as a depth-first recursive pass to pull up appropriate nodes.

Our QP driver should already know how to deal with such a data structure and generate the appropriate tsearch query. However, there will still be limits on the size of individual tsearch queries, so we may not be out of the woods with just that.

Thoughts?