strip accent characters in solr index

[Anand Chitipothu]

Most international records from Library of Congress have titles and authors are in a romanized form with accent characters. It is impossible for people to find these records from search unless we add the accent-less version of the names and titles to solr index.

For example: http://upstream.openlibrary.org/authors/OL617A/Śrī_Śrī
This author name when written in English becomes "Sri Sri". I knew that there exists an entry about this author in OL and still it took me more than one hour to find this record.

[Edward Betts (edwardbetts)]

http://openlibrary.org/authors/OL617A has "Sri Sri" as an alternative name, so it matches. We need another example

[Anand Chitipothu (anandology)]

On 23-Jun-10, at 10:29 PM, Edward Betts wrote:

> http://openlibrary.org/authors/OL617A has "Sri Sri" as an alternative
> name, so it matches. We need another example

http://openlibrary.org/authors/OL204764A/Yarraṃśeṭṭi_Śāyi

[Edward Betts (edwardbetts)]

http://openlibrary.org/search?q=Yarramsetti+Sayi should produce the same results as http://openlibrary.org/search?author=OL204764A

[Edward Betts (edwardbetts)]

Looks like the answer is ASCIIFoldingFilterFactory:

http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.ASCIIFoldingFilterFactory

[Edward Betts (edwardbetts)]

We need to rebuild the solr index to fix this bug.