Merge and Overlay Functions should use Merge Profiles

[Jennifer Pringle]

Wishlist

Evergreen 2.2.5
Postgress 9.1.7
OS: Ubuntu Precise

Currently when merging multiple records within Evergreen or overlaying existing records with new records from a Z39.50 search the lead record is chosen and nothing from the non-lead record(s) is preserved. It would be very useful to have ability to preserve and/or replace certain fields in the MARC record (like when importing MARC records) so that a true merge is performed.

Ideally when you merge or overlay a record you should have to chose a merge profile. These merge profiles would be the same as used in the MARC batch Import/Export (Vandelay).

Existing workflow for merging with modification:
1. Add records to record bucket (http://docs.sitka.bclibraries.ca/Sitka/current/html/bucket-adding-records.html)
2. Click Merge All Records
3. Select the lead record
4. **Select the Merge Profile to be used from the drop down menu (new functionality)**
5. Click Merge

[Sarah Childs (sarahc)]

I agree that being able to merge fields of the MARC would be really useful, and the Vandelay profiles would be a great starting place. To take it another step it would also be useful if you could select fields from the records not selected as the lead record on a case by case basis for fields where you may not always want to retain them.

[Galen Charlton (gmc)]

I will be implementing this, per the following specifications:

http://yeti.esilibrary.com/dev/pub/techspecs/broaden-use-of-merge-overlay-profiles.html

[Galen Charlton (gmc)]

A series of patches is available in the user/gmcharlt/lp1117808_extend_use_of_merge_profiles branch in the working/Evergreen repositories:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmcharlt/lp1117808_extend_use_of_merge_profiles

Please note that this patch also includes the patches for bug 1663663. While that bug is not a strict dependency for this feature, applying those patches is recommended to avoid confusion.

[Kathy Lussier (klussier)]

Hi Galen,

I'm just starting to take a look at this branch. I performed the following steps:

I created a new merge/overlay profile called kathytest.The overlay profile is owned by the Consortium.
I configured the OU setting to default to the stock Full Overlay merge/overlay profile. This setting is set to the Consortium.

When I use the Z39.50 interface or merge the records, the widget does not default to the Full Overlay profile. When I click the dropdown, I only see the Match-Only Merge option. The Full Overlay and kathytest options are not available.

[Galen Charlton (gmc)]

As noted (thus far) in a couple of the commit messages, profiles that have a preserve specification set (like Full Overlay and (I assume) your kathytest) are intentionally omitted from the selector. It's a wart that such profiles can be selected in the library setting.

Here's why such profiles are excluded (and it took a while to work this out during testing with BCLC): in the case of full overlay, it's superfluous; in the case of ones that preserve fields, but aren't full overlay, they're confusing.

In the first case, if all you want to do is fully overlay a record, you don't need a profile. In the record bucket interface, you'd just select whatever lead record you want to run with. In the Z39.50 interface, the record coming from Z39.50 is the target of the merge -- and requesting a full overlay from the incumbent record obviates the point of doing the Z39.50 search in the first place.

In the second case, preserve-specification merge profiles have the effect of flipping the target of the merge, which would be confusing: both the dialog boxes have the cataloger explicitly specify the profile.

All of that said, it does suggest to me that it might be good to add an option to the merge profile selectors to always be able to select /no/ merge profile.

Thoughts?

[Kathy Lussier (klussier)]

Thanks Galen! I obviously didn't read the commit messages closely enough. The intended behavior is different than I expected. I'm going to look at this a little more closely to better understand it before I respond to your question.

[[e-mail] (tji)]

"it does suggest to me that it might be good to add an option to the merge profile selectors to always be able to select /no/ merge profile."

I think libraries wishing to merge without using a profile can create a dummy profile (not dealing with any tags). Those libraries that do not allow this behaviour should not create such a profile.

[Kathy Lussier (klussier)]

The source of my confusion was that I thought the profiles were being applied to the existing record, not the incoming record for Z39.50, since this is the behavior we see in Vandelay. It feels like it's working backwards, but, now that I know what it's doing, I see that it is working as expected. I also see that the selector is defaulting to the profile configured and the library settings and that it remembers the previously-selected profile (we love sticky options at MassLNC!)

I'll defer to Tina's judgement on the question of whether we need an option to select /no/ merge profile. Now that I understand the behavior, I don't think it's necessary, and, if we find out through production use that we need something like that, we can address it at that time.

I just have one question before merging it. I don't know if this was one of the intended use cases, but I used a merge profile that added the 590 tag as a way to preserve the local note. When importing a record via Z39.50 or merging records, I noticed the 590 tag was placed at the top of the record rather than among the other 5xx tags. Is there a way it could be placed in a more logical location in the record?

Thanks Galen!

[Galen Charlton (gmc)]

> I just have one question before merging it. I don't know if this was one
> of the intended use cases, but I used a merge profile that added the 590
> tag as a way to preserve the local note. When importing a record via
> Z39.50 or merging records, I noticed the 590 tag was placed at the top
> of the record rather than among the other 5xx tags. Is there a way it
> could be placed in a more logical location in the record?

Yes, but I think that should be the topic of a separate bug or
enhancement report, as this is an issue with the Vandelay field-level
merge profiles in general.

[Kathy Lussier (klussier)]

Thank you Galen! Also, many thanks to BC Libraries Cooperative for funding this work! Merged to master for inclusion in 2.12.