Wire up aspell for pgtexindex in staging/production

[Paul Everitt]

If this needs to be part of the buildout, or listed in dependencies in README.txt, do that as well.

[Chris Rossi (chris-archimedeanco)]

I've played around with this some on my laptop but I've been left a bit skeptical. The PostgreSQL documentation makes the following claim:

"The Ispell dictionary template supports morphological dictionaries, which can normalize many different linguistic forms of a word into the same lexeme. For example, an English Ispell dictionary can match all declensions and conjugations of the search term bank, e.g., banking, banked, banks, banks', and bank's."

In practice, I've been unable to tease out an example where use of the Ispell dictionary actually yields a result we couldn't get with the algorithmic snowball stemmer which is in place already by default. The "bank" example above, for example, is easily handled by the algorithmic stemmer. A more interesting example would be the verb "to break" since it has some irregular forms that can't be handled by the algorithmic stemmer. Using the ispell, dictionary, however, the lexeme returned for "broken" is "broken", not "break" as you would expect as the result of morphological normalization ("broken" is the past participle of "to break").

I also examined the Ispell dictionary files themselves and they contain only lists of words--I see nothing related to morphology. So I don't have any notion how Postgres could perform morphological normalization using an Ispell dictionary.

There may be something I'm missing, so I won't close the ticket yet, but for the moment the claims in the Postgres documentation seem to be trumped up, at least for the English ispell dictionaries I was able to track down.