Tpac - record details retrieval can be very slow

[Ben Shum]

Evergreen version: ~2.2-beta2

With Tpac, we've noticed that bib record retrieval can be extremely slow when retrieving a bib associated with a results set larger than 2k hits. The initial search results can come up reasonably fast for us regardless of how many hits we get, but clicking on a given record ends up being extremely slow.

Examples from our production Tpac of more than 10k hits:

A search for "dogs" : http://acorn.biblio.org/eg/opac/results?fi%3Aitem_type=&query=dogs&qtype=keyword&locg=1&sort=

A search for "cats" : http://acorn.biblio.org/eg/opac/results?fi%3Aitem_type=&query=cats&qtype=keyword&locg=1&sort=

If you click on a record from that set, it takes ages to load, but when it finally does, things appear cached and "faster". The initial load time is ridiculously bad though.

I've discussed this briefly in IRC with senator, gmcharlt, and others, and the current hypothesis is that the differing method of results retrieval vs. record retrieval is the problem. And record retrieval being slower given the number of hits that come up.

[Lebbeous Fogle-Weekley (lebbeous)]

For reference, the work committed from bug #988146 should be helpful in starting to pinpoint the exact problem. Which we're pretty sure at this point is either the way we're calling unapi.bre or something to do with missing the search that's just been cached from the results page when looking for it from the record detail page.

[Bill Erickson (berick)]

To test the search cache key mismatch theory, Ben could you perform a new search (that's not already cached) and subsequently jump to a record detail page, then grep for "staged search: cached with" in osrfsys log? (It's logged at INFO). You should see something like:

staged search: cached with key=open-ils.search_b9de52bcedc2bd33a50e18accadbb55f, superpage=0, estimated=, visible=98

If the record detail page is re-running and re-caching the search, there should be 2 entries, each with a different cache key.

[Bill Erickson (berick)]

Step 1 to a two-step process pushed to collab/berick/tpac-record-detail-paging-fix

This fixes the speed issue, but creates a bug with the staff client "Last" record button. I'll look at that next after verifying this code is sane.

[Ben Shum (bshum)]

Noting on the bug ticket that we tested berick's branch for paging and it is significantly improved for speed in record retrieval.

Will test part 2 of the resulting issue when ready.

Thanks!

[Bill Erickson (berick)]

I rebased the 2 original commits and pushed another commit to address part 2 (staff client End link). This should take care of it.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/berick/tpac-record-detail-paging-fix

[Ben Shum (bshum)]

Tested both part 1 and 2 of this and it does resolve the record retrieval slowness and it's much snappier now.

For part 2, the End link works, but still can be slow depending on the number of record hits.

Still worth merging for now.

Signed off at: user/bshum/tpac-record-detail-paging-fix

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/bshum/tpac-record-detail-paging-fix

[Lebbeous Fogle-Weekley (lebbeous)]

Merged to master and rel_2_2 with thanks to Ben, Bill and Mike R.