get rid of searchable index
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
shellsink |
Confirmed
|
Medium
|
Joshua Cronemeyer |
Bug Description
This suggestion came from Nick Johnson in the google group. It merits further investigation:
You're currently using the searchable index extension, which has a
couple of shortcomings: It's designed for english text (so it has
lists of stopwords etc that really don't apply here), and it requires
custom indexes, which as you've observed, restrict you to searching on
only one or two terms, and creates indexes that take up large amounts
of space.
However, if you're prepared to forgo efficient sorting of search
results, there is another option: Index stuff yourself, by applying
your own tokenizer to the command lines passed in and concatenating
that with the list of tags into an 'index' field. Then, you can use
App Engine's built in support for merge-joining simple queries. That
is, you can specify any number of equality filters on an entity, and
as long as you don't use inequality filters or sort orders, App Engine
will satisfy the query for you without requiring a custom index. This
will increase the flexibility of search, and massively decrease the
storage overhead you're experiencing.
The disadvantage of this approach is that you can't choose a sort
order, and the natural sort order is based on keys, so it will return
the oldest entries first (unless you're using key names). I don't
think this should be a huge problem, however - most people, when they
search command histories, are looking for a very limited number of
results.
Changed in shellsink: | |
assignee: | nobody → joshuacronemeyer |
importance: | Undecided → Medium |
status: | New → Confirmed |