Add 264 tag values to Record Summary

[Jason Stephenson]

We added 264 information for the Producer, Distributor, Manufacturer, and Copyright information of the record to our record summary. This is in addition to the publisher information that is already displayed.

Thought we'd share this with the community as a RDA-related enhancement to the TTOPAC.

[Jason Stephenson (jstephenson)]

Pushed to a collab branch:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/dyrcona/lp1304462_addl_record_summary_info

Dan Scott mentioned looking into adding schema.org attributes to this branch in IRC.

[Jason Stephenson (jstephenson)]

From the commit message:

This pulls information from the 264 tag where the indicator 2 has
    the following values and displays the information below the Publisher
    info:

    0 - Producer (a,b,c)
    2 - Distributor (a, b, c)
    3 - Manufacturer (a, b, c)
    4 - Copyright (c)

[Yamil (ysuarez)]

Here are the comments I got from one of the catalogers here on a EG 2.5.1 test server, thanks for submitting this code...

- "Looks good. "

- The cataloger then looked at how the search results displays the 264 info when the "Show More Details" feature in search results is active (which I have set to be always on in our system).

Then she added... "It looks like "Publisher" on the search results just pulls the first 264, which in the case of Amour is "Distributor". It would be nice if that label changed based on the indicator number."

<snip>

245 1 0. ‡aAmour / ‡cLes films du losange, X Filme Creative Pool, Wega Film présentent une coproduction Franco-Germano-Autrichienne ; en coproduction avec France 3 Cinéma, Ard Degeto, Bayerischer Rundfunk, Westdeutscher Rundfunk ; avec la participation de France Televisions, Canal+, Cine+, ORF Film/Fernseh-Abkommen ; produit par Margaret Menegoz, Stefan Arndt, Veit Heiduschka/Michael Katz ; un film de Michael Haneke.
246 1 . ‡iEnglish title: ‡aLove
257 . ‡aFrance ‡aGermany ‡aAustria. ‡2naf
264 2. ‡aCulver City, Calif. : ‡bSony Pictures Classics, ‡c[2013]
264 4. ‡c©2013.

</snip>

[Jason Stephenson (jstephenson)]

My code doesn't change what the Publisher field does. I simply added the others.

If it turns out that publisher is grabbed the first field rather than by indicator, I could change that, but I wonder if that should be done in this branch or another.

[Dan Scott (denials)]

Are there any RDA records to test this with in the sample set? If not, can we get some added as part of this bug? That would _really_ help testing this :)

[Yamil (ysuarez)]

Dan: I currently only have records from OCLC, which I don't think I can post/share in their entirety, unless I am wrong. I can ask one of the catalogers here to help me create a dummy RDA record for testing purposes.

In the meantime here are some snippets from the other two records I used for illustration purposes...

245 0 0. ‡aLeonard Bernstein's Young people's concerts : ‡bwith the New York Philharmonic. ‡nVolume 2 / ‡cCBS Entertainment ; the Leonard Bernstein Office, Inc.
246 3 0. ‡aYoung people's concerts
246 3 0. ‡aYoung people's concerts with the New York Philharmonic
264 1. ‡aWest Long Branch, NJ : ‡bKultur Video, ‡c[2013]
264 2. ‡aWest Long Branch, NJ : ‡bDistributed by Kultur International Films, Ltd.
264 4. ‡c©2013
300 . ‡a9 videodiscs (approximately 27 hr.) : ‡bchiefly black and white, some color ; ‡c4 3/4 in. + ‡e1 booklet (18 pages : illustrations ; 18 cm.)

245 0 0. ‡aInside Llewyn Davis : ‡boriginal soundtrack recording.
264 1. ‡a[New York] : ‡bNonesuch, ‡c[2013]
264 4. ‡c℗2013
300 . ‡a1 audio disc : ‡bdigital, CD audio ; ‡c4 3/4 in.

[Yamil (ysuarez)]

Jason (and others):

It is up to you if you want to throw something in this same working branch or just keep your branch as is. If you want I can sign off on it as it stands right now, though I only tested in on EG 2.5.1.

I can also try to create a collab branch where I try to take a stab at improving what I am seeing in the search results. I can also make a new LP bug for it.

Just to recap for those reading along, for the DVD bib record for "Amour" it lists the following in the full bib view, taken from the 264 tags (see below for 264 MARC data)...

-------

Record details

Physical Description: 1 videodisc (approximately 127 min.) : sound, color ; 4 3/4 in.
Distributor: Culver City, Calif. : Sony Pictures Classics, [2013]
Copyright: ©2013.

--------

but in the search results, after clicking on "show more details" it shows....

----------
 Projected medium
Publisher: Culver City, Calif. : Sony Pictures Classics, [2013]
Phys. Desc.: 1 videodisc (approximately 127 min.) : sound, color ; 4 3/4 in.

-----------

264 data...

245 1 0. ‡aAmour / ‡cLes films du losange, X Filme Creative Pool, Wega Film présentent une coproduction Franco-Germano-Autrichienne ; en coproduction avec France 3 Cinéma, Ard Degeto, Bayerischer Rundfunk, Westdeutscher Rundfunk ; avec la participation de France Televisions, Canal+, Cine+, ORF Film/Fernseh-Abkommen ; produit par Margaret Menegoz, Stefan Arndt, Veit Heiduschka/Michael Katz ; un film de Michael Haneke.
246 1 . ‡iEnglish title: ‡aLove
257 . ‡aFrance ‡aGermany ‡aAustria. ‡2naf
264 2. ‡aCulver City, Calif. : ‡bSony Pictures Classics, ‡c[2013]
264 4. ‡c©2013.

[Ben Shum (bshum)]

I think I've narrowed down the potential source of the issue that Yamil is seeing. It seems that in results, we're looking to use pubinfo to display a combined statement about publication information. In record, it's broken out into its distinct parts -- pubpace, publisher, and pubdate. The issue is that the hunt for graphic_880s linking is not paying attention to the second indicator flag of any given 264. It seems to grab any one present, which is why Yamil can see a 264 with ind2=2 being used instead of the ind2=1 that we would expect.

Pertinent lines from misc_util.tt2:

args.pubinfo = "$args.pubplace $args.publisher $args.pubdate";

and later:

args.pubinfo = (args.pubinfos.size) ? args.pubinfos.0 : '';

So args.pubinfo is being defined once as the combined pubplace publisher and pubdate, but then defined again as the first in a group of pubinfos taken by looking for all the 260s and 264s in the hunt for linked graphic_880s.

I think what we should do to resolve this issue for search results would be to break it up similarly to how record details is done and have separate parts for pubplace, publisher, and pubdate to be displayed if there is something for publisher. Along the way, maybe we should enhance the results page to display distributor if that's what exists, etc. and get more meaningful 264 entries to display with the right label in search results.

Will try slapping some code together later.

[Ben Shum (bshum)]

Not sure if we should actually pursue the broken 264 publisher display in results in a new bug or expand this one to include bug fixing it.

Also, dbs is right that we don't seem to have any sample bibs in the test dataset that include any 264's. We should pull in some catalogers to help create examples of all the things we need to test for us!

[Dan Scott (denials)]

I've reached out to a possible source for RDA records, as I personally feel that the lack of any test data which we can use to reproduce results and ensure that we're generating what is desired is a blocker for sign-off :/

[Jason Stephenson (jstephenson)]

Yamil: You should be able to share your OCLC records with anyone. My understanding is that once they are in your catalog, that is allowed. If not, then your public access catalog already violates OCLC's alleged copyright or whatever they claim.

On the display in search results vs. summary: My code only touches the summary. The behavior from the search results list was there before I made my changes. Therefore, I think that should be a second bug, but if anyone wants to address it in this bug, feel free to out vote me. I don't really care either way and I don't really care if the results behavior is changed or left as it is.

[Rogan Hamby (rogan-hamby)]

Jason is correct. Now, I wouldn't be surprised if OCLC didn't like it but they can't make a legal claim. If there is a concern on that part I can help dig up and/or create records as well.

Excuse my brevity, sent from my iPhone

> On Apr 11, 2014, at 8:34 AM, Jason Stephenson <email address hidden> wrote:
>
> Yamil: You should be able to share your OCLC records with anyone. My
> understanding is that once they are in your catalog, that is allowed. If
> not, then your public access catalog already violates OCLC's alleged
> copyright or whatever they claim.
>
> On the display in search results vs. summary: My code only touches the
> summary. The behavior from the search results list was there before I
> made my changes. Therefore, I think that should be a second bug, but if
> anyone wants to address it in this bug, feel free to out vote me. I
> don't really care either way and I don't really care if the results
> behavior is changed or left as it is.
>
> --
> You received this bug notification because you are subscribed to
> Evergreen.
> Matching subscriptions: evergreenbugs
> https://bugs.launchpad.net/bugs/1304462
>
> Title:
> Add 264 tag values to Record Summary
>
> Status in Evergreen - Open ILS:
> New
>
> Bug description:
> We added 264 information for the Producer, Distributor, Manufacturer,
> and Copyright information of the record to our record summary. This is
> in addition to the publisher information that is already displayed.
>
> Thought we'd share this with the community as a RDA-related
> enhancement to the TTOPAC.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/evergreen/+bug/1304462/+subscriptions

[Dan Scott (denials)]

Just for the record, the http://oclc.org/worldcat/community/record-use.en.html includes the following *recommendation*:

"""
OCLC, supported by the OCLC Global and Regional Councils and the Board of Trustees, recommends that the Open Data Commons Attribution (ODC-BY) license be used by OCLC members who want to release their library catalogs under an open data license structure.
"""

We could check with the Conservancy to see if this recommendation has any real force (it seems unlikely) and if we did follow the recommendation, whether the ODC-BY license would be compatible with our project's predominant GPLv2+ licensing. There's a thread at debian-legal with some analysis of ODC-BY at https://lists.debian.org/debian-legal/2013/09/msg00047.html

[Yamil (ysuarez)]

On the topic of addressing this issue the Berklee cataloger noticed on this same bug or crate a new one. I would put my initial vote on making a separate bug for the new issue of how the distributor data is showing up in search results. I would also volunteer to start up the new bug. From my testing, the code that Jason wrote worked fine, and I think I may be ready to offer him a sign off. Then again, I am open to suggestions that we should wait to test Jason changes more with additional RDA records.

On the topic of sharing OCLC records, I can bring this up to the board to have the SFC help us out on this. I having been playing it very safe with our OCLC records. I would love to have our community lawyer set us straight.

Finally, I can write to the cataloging list to ask for additional test RDA records. I can initially ask for non-OCLC records until we have spoken to our counsel. Also, I wonder if the groups that are in charge of RDA stewardship has a set of test records they can share with the EG community or the board, etc.

Dan you are right, that access to several RDA records in our shared data sets is key in making progress with adding RDA support in EG.

[Dan Scott (denials)]

My source has promised to supply cleanly redistributable RDA records by next week, if we can wait that long. She has led RDA training sessions and knows her stuff, so I trust her. And I will reveal her identity once the records are in our hands, for the purposes of credit/karma :)

Until then I have access to some records that I can use for spot testing and my own schema.org contributions.

[Ben Shum (bshum)]

Going to continue work reviewing this and fixing bugs with 264.

[Dan Scott (denials)]

For what it's worth, I agree with Ben's assessment in comment 8. The more that we can single-source the display mechanism between record details and search results, the fewer bugs we're going to see.

[Dan Scott (denials)]

Finally signed off on Jason's commit, using the test records that Yamil provided (thanks Yamil!) and some local variations on those records to get "manufacturer" (264 ind2=3), and added a commit for the schema.org properties that we can extract from the exciting new RDA fields. Which turns out to be not much, because RDA is ... well, see the commit message I guess.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dbs/lp1304462_rda_264

[Kathy Lussier (klussier)]

Can someone add a release notes entry too? Thanks!

[Yamil (ysuarez)]

I can whip something up for Jason to review.

[Yamil (ysuarez)]

Here is a simple attempt at a release note that I created with my co-worker Amy for Jason's and Dan's work. Feel free to make edits to it and/or give me the go ahead to push to working.

OPAC

The OPAC now displays RDA bib tag 264 information for Producer, Distributor, Manufacturer, and Copyright within a full bib record’s summary. This is in addition to the RDA bib tag 264 publisher information, indicator 2 equal to 1, that was already being displayed in previous versions of Evergreen. The OPAC full bib view also now contains the Schema.org copyrightYear value.

Thanks in advance,
Yamil

[Jason Stephenson (jstephenson)]

Yamil, the release not looks fine to me. You can put it in the docs any time you want.

[Ben Shum (bshum)]

I signed off on Jason and Dan's work and added a few additional commits to deal with the search results portion of the catalog and making that also support all the variants of tag 264.

The following collab branch contains this work thus far: collab/bshum/lp1304462_rda_264

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/bshum/lp1304462_rda_264

I also included Yamil's release note entry and added a line about how we're fixing this for search results as well.

[Dan Scott (denials)]

Cool, I signed off on all of the commits in this branch, and fixed up the commits over in bug # 1308768, tested the lot of them, and pushed the whole dang thing to user/dbs/lp1304462_rda_264_and_lp1308768_sample_RDA_records_from_loc

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dbs/lp1304462_rda_264_and_lp1308768_sample_RDA_records_from_loc

As far as I can tell, only my commit e9d1bf4a9de0129b (for making the sample RDA records loadable) needs signing off from that whole batch.

[Ben Shum (bshum)]

Thanks for the review Dan. The combined work looked good to me. Pushed to master!

[Yamil (ysuarez)]

Is this something that is worth to be pushed into 2.6? FWIW, I am finally migrating to 2.6 in two weeks and I can try to re-work the code, where/if necessary, to work on 2.6 and can push to a working branch for review. Though this functionality might not be the type of feature that is normally back ported, unlike say major bug fixes. Thanks in advance for humoring my question.