attempting to retrieve an invalid MARC record via Z39.50 can result in unexpected memory consumption

Bug #1358916 reported by Galen Charlton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Fix Released
Undecided
Unassigned
2.5
Fix Released
Undecided
Unassigned
2.6
Fix Released
Undecided
Unassigned

Bug Description

Z39.50 targets in the wild have been known to return MARC records that are invalid by virtue of their being longer than 99,999 octets. In at least one case, attempting to retrieve such a record has resulted in an open-ils.search drone ballooning to 3G+ of memory usage during a stage where XML::LibXML is trying to parse a MARCXML version of the record.

An example can be found by searching for

@attr 1=4 @attr 4=1 @attr 5=1 "three can play that game"

at the target z3950.fcla.edu/RF. As of the date of filing this bug, the second hit in the result set triggers the memory ballooning.

Evergreen master

Revision history for this message
Galen Charlton (gmc) wrote :

A patch is available in the user/gmcharlt/lp1358916_dont_fetch_overlarge_records_via_z3950 branch of the working/Evergreen repository:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmcharlt/lp1358916_dont_fetch_overlarge_records_via_z3950

tags: added: pullrequest z3950
Revision history for this message
Galen Charlton (gmc) wrote :

The patch is now in production at one consortium and appears to have the desired effect. An open question is whether to adjust the hit count to exclud rejected records.

Revision history for this message
Ben Shum (bshum) wrote :

Pushed to master and backported to rel_2_6 and rel_2_5 as a bug fix. Thanks Galen! Silly MARC....

Changed in evergreen:
milestone: none → 2.7.0
status: New → Fix Committed
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.