Script to convert pgtextindex to zope.textindex

[Chris Rossi]

We have a script that converts from the zope text index implementation in repoze.catalog to the pg based implementation in repoze.pgtextindex. We also need a script that can convert the other way in case we need to bail out.

[Paul Everitt (paul-agendaless)]

Let's make M54 more about its original purpose. If you have time in M54 to work on M55 stuff, so be it.

[Chris Rossi (chris-archimedeanco)]

My plan for this is take the 'use_pgtextindex' script and turn it into a 'reindex_text' script. This script would, by default, rebuild a text index of the same type already in use or if specified by a flag (--pg or --zope) will build a new text index of the type specified, allowing the user to switch arbitrarily between text index types.

The difficult part, such as it is, has already been written. The script already builds a new text index in parallel with the existing one so as not to interrupt normal site operations. It also indexes documents in batches using several transactions to prevent overconsumption of memory on large sites like OSF. An upshot of this approach is the script can be interrupted and resumed.

So the only new work is just renaming the script and adding the options to create either a zope text index or a pg text index when doing the reindex. Should be a very quick job.

[Paul Everitt (paul-agendaless)]

Shane, I'll hand this one to you for this week. But this needs to get done. If you don't plan to do KARL work this week, please let me know.

[Shane Hathaway (shane-hathawaymix)]

I implemented this just the way Chris suggested, but I don't have commit privileges to the karlserve project so I made a fork and a pull request:

https://github.com/karlproject/karlserve/pull/2

[Paul Everitt (paul-agendaless)]

Assigning to Chris to do the merge. I'm making this a high priority as it is the one thing that must happen before we can go live.