Add support for (nearly) direct access to the full unapi backend

Bug #1367926 reported by Mike Rylander on 2014-09-10
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Evergreen
Wishlist
Unassigned

Bug Description

Some parts of Evergreen (notably the TPAC), and various 3rd party services, would benefit from full and direct access to the power of the backend (read: faster, more complete, more API-stable) unAPI (um) API. Related, many moons ago, I built a perl module (OpenILS::Utils::TagURI) to make parsing of tag URIs systematic for use in OpenSearch and unAPI contexts. This branch uses O::U::TagURI and a simple shim inside open-ils.supercat to expose the power of that backend unAPI functionality.

Code coming soon. Watch this space...

Kathy Lussier (klussier) wrote :

I took a look at this code on a MassLNC Sandbox this morning. I was able to retrieve data using the copy's barcode and played around with adjusting which bits of data should be retrieved.

However, I had trouble retrieving output by record ID.

Using the example MIke included in his git branch, I tried the following URL:
http://mlnc2.mvlcstaff.org/opac/extras/unapi?tag::U2@bre/167{holdings_xml,acn,acp,mra}&format=mods32

I get then get the following message: Sorry, we couldn't a record with the id of in format mods32.

This message appears with any format I try to use.

I ultimately was able to successfully retrieve output using the following URL (note the addition of id= after the question mark):
http://mlnc2.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/167{holdings_xml,acn,acp,mra}&format=marcxml

However, marcxml is the only format that works with the above URL. If I try mods32, which was the format used in Mike's example, I get an XML parsing error:not well formed.

Also noting that this branch needs a release notes entry, but I can add one when I sign off on it. I just want to verify what the correct URL format should be for the bre example and if the mods32 format (or others) should be working here.

Thanks Mike!
Kathy

tags: added: needsreleasenote
Changed in evergreen:
assignee: nobody → Josh Stompro (u-launchpad-stompro-org)

I don't think this is quite ready, the fact that "format=" should show all the possible formats, but the only format that seems to work is marcxml should be addressed.

unapi?id=tag::U2@acp/FRE410001102{holdings_xml,acn,acp,mra,bre}/-/0/barcode&format=html
returns marcxml

unapi?id=tag::U2@bre/165&format=mods32
XML Parsing Error: not well-formed

The full format looks to be
http://example.com/opac/extras/unapi?id=<classname>/<id>[<paging>]{<includes>}/<location>/<depth>/<pathinfo>?format=

I tried to figure out why only marcxml seemed to be a supported format, but I couldn't figure out where the issue was.

I'm curious why the regex is set to match /-/0/barcode as the pathinfo, I'm assuming it is just a shortcut to leaving location and depth unset if location = "-"?

Josh

Changed in evergreen:
assignee: Josh Stompro (u-launchpad-stompro-org) → nobody
Mike Rylander (mrylander) wrote :

Josh,

For U2 tag URIs, only the bre object type supports differing formats, and the rest expect to have a format of xml. Anything else is for the future. That's a limitation of the underlying unapi DB functions. The format list is an artifact of the version 1 implementation, but we can probably arrange to trim it in the face of a version 2 tag URI for the time being. As for the regex, yes,

Kathy and Josh,

Can you tell where the XML parsing error is coming from? Is it just a browser complaint? Do you still have the branch loaded somewhere that I could use wget or curl to pull from, or, can you pull the result and share it?

I'll work up some release notes soon.

Thanks!

Mike Rylander (mrylander) wrote :

er ... hit send too soon.

... As for the regex, yes, location="-" and depth="0" is the default, meaning "everywhere". "-" is the canonical stand-in for "top of the org tree" in supercat.

RE: XML Parsing Error, in my case it is from Firefox, it just doesn't like the xml, but the content is there. Here is a pastebin of the output that Firefox doesn't like. My test server is still up but isn't accessible externally.

Using the sample DB load data.
/opac/extras/unapi?id=tag::U2@bre/165&format=mods32
http://pastebin.com/N6pa9d7q

Josh

Mike Rylander (mrylander) wrote :

Josh,

xmllint doesn't seem to mind that content, and it's definitely is mods. I think FF is just complaining about the embedded (possibly broken or otherwise non-UTF8 encoded) non-ascii chars. That, of course, would be a data of FF problem.

Thanks for grabbing the output!

Kathy Lussier (klussier) on 2015-08-18
Changed in evergreen:
assignee: nobody → Kathy Lussier (klussier)
Kathy Lussier (klussier) on 2015-08-19
Changed in evergreen:
assignee: Kathy Lussier (klussier) → nobody
Mike Rylander (mrylander) wrote :

I've rebased the branch to master and added the ability to include non-MARC fields from biblio.record_entry. New branch is here:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp_1367926-extended-unapi-rebase-bre_extern

Galen Charlton (gmc) wrote :

I've reviewed and squashed Mike's patches, and the branch collab/gmcharlt/lp1367926-unapi2 in the working/Evergreen repository is now ready for review:

  http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/gmcharlt/lp1367926-unapi2

This branch also addresses the following concerns:

* lack of release notes
* character encoding
* the list of available formats (when given a U2 ID) now includes only the applicable ones

It also adds some unit tests.

tags: removed: needsreleasenote
Kathy Lussier (klussier) wrote :

Thanks for the new branch Galen! This looks good.

I have a few questions:

1. For the release notes, where you explain how to form the URL, should we include something that says &format=xxxx should be tacked on to the end of the URL? I can make the changes to the release notes, but I wasn't sure if it really was needed since I'm unclear on whether the format= is necessary. I do see that, if it's missing, the available formats are returned as you described in your above comment.

2. I'm not sure if this is a bug or a problem with my URL. I was trying some of the different class names and encountered problems when I tried cbs. Using this URL:

http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@cbs/3&format=mods32

I get a bib source of Project Gutenberg instead of the correct bib source, oclc. When I use the class in an include, it properly lists the bib source as OCLC. See

http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/3{holdings_xml,acn,acp,mra,cbs}&format=mods32

Is this a bug or user error?

> 1. For the release notes, where you explain how to form the URL, should
> we include something that says &format=xxxx should be tacked on to the
> end of the URL? I can make the changes to the release notes, but I
> wasn't sure if it really was needed since I'm unclear on whether the
> format= is necessary. I do see that, if it's missing, the available
> formats are returned as you described in your above comment.

Yeah, a note about the format parameter would be good to have in the
release notes.

> 2. I'm not sure if this is a bug or a problem with my URL. I was trying
> some of the different class names and encountered problems when I tried
> cbs. Using this URL:
>
> http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@cbs/3&format=mods32
>
> I get a bib source of Project Gutenberg instead of the correct bib
> source, oclc. When I use the class in an include, it properly lists the
> bib source as OCLC. See
>
> http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/3{holdings_xml,acn,acp,mra,cbs}&format=mods32
>
> Is this a bug or user error?

An opportunity for documentation clarification. :)

"tag::U2@cbs/3" signifies that you want *just* the config.bib_source
row whose ID is 3.

Kathy Lussier (klussier) wrote :

I added a commit to the release notes to add the format to the URL and merged it to master.

Thanks so much for your work on this Mike and Galen!

Changed in evergreen:
status: New → Fix Committed
milestone: 2.next → 2.10-beta
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers