Copy Import Development

[Liam Whalen]

Copy Import Development (in Vandelay)

There are 2 pieces of development required in the Batch Loader (Vandelay) in Evergreen to help us with loading of on-order and full MARC records for our Sitka libraries. Both of these options fit logically in the new section in 2.6 in MARC Batch Import/Export called ‘Copy Import Actions’, where a new 2.6 feature called ‘Auto-overlay In-process Acquisitions Copies’ is now available. This development will add 2 new features:

1)Auto-overlay On-order Cataloguing Copies

2) Use Org Unit Matching in Copy to Determine Best Match

Note, these options will also need to be offered on the secondary screen from the batch load queue, when you choose “Import Selected Records’ or ‘Import All Records”.

1. Auto-overlay On-order Cataloguing Copies

This development is conceptually similar to the new ‘Auto-overlay In-process Acquisitions Copies’, but we need it to work for copies that were not created from an acquisitions workflow. We need to be able to load a file of full MARC records with full permanent holdings information, and to be able to overlay this holdings information over an existing On order copy record for the same org unit. For the match and overlay on the copy to happen, the following criteria must be met:

a) There must be an attached copy to the record that has an On order status.

b) The owning library for the existing On order copy must match the incoming value for owning library as defined in the Holdings Import Profile.

c) If there is more than one On order copy for a particular owning library attached to the record, look to the copy information as defined in the Holdings Import Profile to find a match on the existing record (copy 1 to copy 1). If no copy information is sent in the incoming file, then, use the earliest create date on the On order record to determine which one to overlay. It is possible that there will be multiple holdings statement for the same org unit attached to one record in the incoming MARC file; if so then match one to one (eg. first holdings import field to first On order record, second to second etc.).

d) Because the incoming full item information will also include a call number, most of the time we will be overlaying on the volume record as well. However, there is the possibility that an existing volume record will have more than one copy, and only one of the copies will be replaced. In this case, we will want to update the volume record with the call number, but only update the copy records if the holdings information are received for them. In other words, the volume record will have the new call number (typically the brief on-order record will have ON ORDER as the call number which will be replaced), and there may be one copy record that has full information overlaid, but another copy may remain as status of ‘on order’, with no other information overlaid.

2) Use Org Unit Matching in Copy to Determine Best Match

A challenge with automatic/machine driven matching, is how to select which match to use, when more than one match is found. Vandelay can determine a match score for this purpose, but that match score is dependent on data MATCHING between the existing and incoming record. This works well for matching on a control number such as ISBN. However we need another criteria to automatically determine which record amongst a set of matched records to use when the match score is the same. It is not uncommon that in a batch load you can have 4 to 5 matches per ISBN, so all of these records can have the same matchscore. In this case, we want the software to look to the value of the owning library of the attached volumes/copies, and choose the record that has the most copies where the owning library matches the owning library as defined in the Holdings Import Profile . If no records match the owning library, then choose the record that has the most owning libraries in the incoming’s record federation. If there is still no match, then default to the behaviour the system currently uses for determining best match when all matched records have the same match score.

Note: If a best match is checked off manually in the interface, we want the manual check to override the ‘Use Org Unit Matching in Copy to Determine Best Match’ option, if checked.

[Liam Whalen (whalen-ld)]

Here is my working branch for this code:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/ldw/RT21151_to_LP1444644

[Jeff Davis (jdavis-sitka)]

I haven't tested this branch against current master, but it's a clean cherry-pick and we've been using the new feature heavily at Sitka in the four years since it was first developed. It would be nice to see this functionality made available in the new Angular Vandelay UI.

[Jane Sandberg (sandbergja)]

This looks very cool, Liam and Jeff. Since this adds some new features, it will need a release notes entry. Also, now that the Angular Vandelay interface has been merged, the UI pieces of this branch should now be added to the Angular Vandelay interface, rather than the dojo one.

[Jeff Davis (jdavis-sitka)]

I am working on adding this feature to Angular.

[Jeff Davis (jdavis-sitka)]

Removing the pullrequest tag for the moment since a few changes are needed. I do still plan to have this ready for 3.4 feature slush.

[Jeff Davis (jdavis-sitka)]

Revised for Angular compatibility and to avoid making assumptions about org tree structure:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/jeffdavis/lp1444644-vandelay-copy-import-squashed

[Galen Charlton (gmc)]

Pushed to master for inclusion in 3.4. Thanks, Liam and Jeff!

[Jeff Davis (jdavis-sitka)]

See bug 1845260 for a follow-up fix.