Deep dive on FTS, PG 9.6, and rum

[Paul Everitt]

Since November (and into the stuff that's imminent), we have a lot of improvements at the storage level. As we convert more views to jsonindex, we'll eliminate most of the hot spots.

"Live Search" remains a problem area:

- PG FTS does relevance ranking with a full table scan of compressed TOAST files

- LiveSearch is pathological...we search for any document with a word starting with 3 or more typed in characters...which usually means a LOT of matched documents to be relevance ranked

- We then do that query 6 times...once for each LiveSearch "group" (although it winds up being the same total for relevance ranking)

In this task, learn a lot about FTS in PG and specifically come up with a reproducible measurement scheme that you're comfortable with. Then, explore how to do better, focusing on the rum extension.

[Jim Fulton (jim-zope)]

I think this is done, but there doesn't a status for: we looked and it didn't work out.