OPAC issn search fails

[George Duimovich]

EG 2.0.2
PG 8.4

Bib record with MARC 022 data is as follows:

022 ‡a0149-1423

If I perform a search for ISSN "0149-1423" I get the 1 hit I expect; however, if I search for "01491423" I get 0 hits.

Expected behavior: ISN searches should not be punctuation sensitive.

Same behavior in our 1.6.1.4 prod system.

[Dan Scott (denials)]

Please give us more information - how are you performing the search? By keyword? By the "advanced search dojo widget" identifier.issn index? By the Advanced Search page, left-hand "ISSN" search box?

This information will help us narrow down the problem.

[George Duimovich (george-duimovich)]

Oh yeah - you guys can't read minds! Sorry about that - lots going on here..

Advanced Search page, left-hand "ISSN" search box would be the one.

[Dan Wells (dbw2)]

After a good bit of research, here is what I can say about the state of ISxN searching in Evergreen 2.0.

First, some history. ISxN searching, as exposed via the OPAC, uses two API methods: "open-ils.search.biblio.issn", and "open-ils.search.biblio.isbn". These are both simple wrappers for 'open-ils.storage.id_list.biblio.record_entry.search.issn', and 'open-ils.storage.id_list.biblio.record_entry.search.isbn', respectively, and these two methods share a single subroutine as their implementation, 'isxn_search()'. This subroutine is very simple, doing an SQL 'LIKE' search on the value fields in metabib.full_rec table, filtering on tag '022' or '020/024' as required. This subroutine was also coded to match certain general normalizations which were expected of the metabib.full_rec values, but not specific to these fields.

In 2.0, many aspects of normalization were made more robust. One result of these changes is that standard number (020, 022, 024) fields are no longer being normalized in metabib.full_rec. Since the above API methods are expecting normalized data in those fields, they no longer work. Adding to the confusion is the fact that the current 2.0 upgrade practice does NOT suggest a full re-ingest, so this means that the API methods will work for any records brought over from 1.6 until they are edited (or otherwise reingested) for the first time. Since the methods use 'LIKE' underneath the surface, a temporary workaround is to use a '%' for any '-' or space which may be in your number (e.g. '0123%1234').

Going forward, though, we now have 'identifier' indexes which can be custom populated AND custom normalized to meet the needs of the identifiers in question. These are already defined for both ISSN and ISBN in 2.0, though a different bug affects these indexes currently (more on that in a moment). You can search these indexes using the 'Advanced' button (which is not the same as the 'Advanced' search screen, neither on the surface nor underneath).

Eventually, it is clear that the 'Advanced' search screen and the 'Advanced' search button should not be doing two different things, and that the Advanced search screen should be taking advantage of the new infrastructure. For the sake of expediency, I am going to advocate that this happen in two phases. First, we will make the "open-ils.search.biblio.issn" and ".isbn" API methods be wrappers of "open-ils.search.biblio.multiclass.query". This will ensure that these APIs continue to function for anything which might be using them, the OPAC included. However, upon further inspection, these APIs were never really appropriate for use in the OPAC, as they do not directly support any kind of paging of results, so the entire result set of IDs gets downloaded for every page view. Under normal circumstances, this weakness is mitigated by the fact that ISxN searches generally have a small result set, most often a single result, so changing this need not be a particularly high priority. At some point, though, the Advanced search screen widget should be updated to function like the Advanced search button, and initiate a full-featured multiclass search.

I expect to commit some changes fo...

[Dan Wells (dbw2)]

The 'phase 1' fix as noted in my comment above is here:

http://svn.open-ils.org/trac/ILS/changeset/19922

The 'params' related bug will be handled separately.