Import z3950 - using local catalog search option results in network error

Bug #1290496 reported by Ben Shum on 2014-03-10
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Evergreen
High
Unassigned

Bug Description

Evergreen master

When using the import z3950 interface and selecting the option for "local catalog" as a search source, the interface will do its search dance, but eventually time out with a network error. Something like:

Network or server failure. Please check your Internet connection to demo.biblio.org and choose Retry Network. If you need to enter Offline Mode, choose Ignore Errors in this and subsequent dialogs. If you believe this error is due to a bug in Evergreen and not network problems, please contact your help desk or friendly Evergreen administrators, and give them this information:
method=open-ils.search.z3950.search_class
params=["0fa936fdd5188ed3a9948d9a00dcb300",{"service_array":["native-evergreen-catalog"],"username_array":[""],"password_array":[""],"limit":10,"offset":0,"search":{"title":"Star Trek"},"service":["native-evergreen-catalog"],"username":[""],"password":[""]}]
THROWN:
null
STATUS:

Investigations ongoing.

Jason Stephenson (jstephenson) wrote :

This should be a 2.6 release blocker IMHO. I've targeted it at 2.6.0-RC1. I'd say it could slide there but definitely needs fixed before 2.6.0 final.

Changed in evergreen:
status: New → Confirmed
importance: Undecided → High
milestone: none → 2.6.0-rc1
Jason Stephenson (jstephenson) wrote :

Trying the above query in srfsh returns nothing for me. It should return something. It also takes over 60 seconds to return that nothing, so that explains the timeout. I will dig deeper, because I think the problem is withing open-il.storage.biblio.multiclass.search_fts.

srfsh# request open-ils.search open-ils.search.z3950.search_class "9175a753c9976db5616101a6b03f56fe" {"service_array":["native-evergreen-catalog"],"username_array":[""],"password_array":[""],"limit":10,"offset":0,"search":{"title":"Star Trek"},"service":["native-evergreen-catalog"],"username":[""],"password":[""]}

Received Data: {
  "records":[

  ],
  "service":"native-evergreen-catalog"
}

------------------------------------
Request Completed Successfully
Request Time in seconds: 71.847174
------------------------------------

Jason Stephenson (jstephenson) wrote :

Dunno. None of the code that seems to be the problem has changed since Dan Wells did a whitespace clean up in June of 2013.

My assumption is the problem is related to some database changes that somehow affect z39.50 search of the local catalog, but not regular search. I could be wrong. Wouldn't be the first time.

Ben Shum (bshum) wrote :

In speaking with some Sitka folks, we fear that local catalog matching for acquisitions may also be affected when trying to load records causing line items to not be linked up with hits for existing bibs. Will test further tomorrow myself, but agree that this could be a major issue for staff on multiple fronts now.

Jeff Godin (jgodin) on 2014-03-19
Changed in evergreen:
assignee: nobody → Jeff Godin (jgodin)
Jeff Godin (jgodin) on 2014-03-19
Changed in evergreen:
assignee: Jeff Godin (jgodin) → nobody
Ben Shum (bshum) wrote :

So based on Bill and Dan's looking around, we identified that this may be a result of the zsearch database call running too long. Likely a result of joining against metabib.rec_descriptor in the joins and that being a new view.

Need to poke more at this.

The acquisitions issue is unrelated and a new bug may be filed.

Dan Wells (dbw2) wrote :

Here is a workaround branch:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dbwells/lp1290496_z3950_search

Long term problems will still need to be addressed. From the commit:

1) Should we rewrite parts of open-ils.search.biblio.multiclass (and the underlying storage methods) to use the new attr structures more directly, rather than through a view of a view? (rec_descriptor / record_attr)

2) Alternately, should we rip it out? If so, we will need to also remove/nullify the "use_staged_search" option, since there will be no functional non-staged offering.

Jason Stephenson (jstephenson) wrote :

Dan's branch seems to work for me and Ben Shum reports that it also works for him.

I hesitate to push this right away because a) it lacks a pullrequest tag and b) because Dan asks for some discussion of the two points raised in the previous comment.

While I am all for better long term solutions, my main concern is there time to work one of those two solutions out before release? If not, I think we should go ahead and push this branch.

Dan Wells (dbw2) wrote :

Missing pullrequest was just a late night oversight. The discussion points are for later, please feel free to push.

tags: added: pullrequest
Jason Stephenson (jstephenson) wrote :

I have pushed from bshum's signoff branch, so we got 3 sign offs on this one! :)

Changed in evergreen:
status: Confirmed → Fix Committed
Mike Rylander (mrylander) wrote :

IMO, the patch as offered is enough for this bug. The native Z search wrapper should really get an overhaul, but that's broader than the reported problem.

I agree with Jason.

Jennifer Pringle (jpringle-u) wrote :

JFYI this fix fixes the same problem in the acq MARC Federated Search too.

Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers