get rid of searchable index

Bug #339542 reported by Joshua Cronemeyer
2
Affects Status Importance Assigned to Milestone
shellsink
Confirmed
Medium
Joshua Cronemeyer

Bug Description

This suggestion came from Nick Johnson in the google group. It merits further investigation:

You're currently using the searchable index extension, which has a
couple of shortcomings: It's designed for english text (so it has
lists of stopwords etc that really don't apply here), and it requires
custom indexes, which as you've observed, restrict you to searching on
only one or two terms, and creates indexes that take up large amounts
of space.

However, if you're prepared to forgo efficient sorting of search
results, there is another option: Index stuff yourself, by applying
your own tokenizer to the command lines passed in and concatenating
that with the list of tags into an 'index' field. Then, you can use
App Engine's built in support for merge-joining simple queries. That
is, you can specify any number of equality filters on an entity, and
as long as you don't use inequality filters or sort orders, App Engine
will satisfy the query for you without requiring a custom index. This
will increase the flexibility of search, and massively decrease the
storage overhead you're experiencing.

The disadvantage of this approach is that you can't choose a sort
order, and the natural sort order is based on keys, so it will return
the oldest entries first (unless you're using key names). I don't
think this should be a huge problem, however - most people, when they
search command histories, are looking for a very limited number of
results.

Changed in shellsink:
assignee: nobody → joshuacronemeyer
importance: Undecided → Medium
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.