Add support for (nearly) direct access to the full unapi backend

[Mike Rylander]

Some parts of Evergreen (notably the TPAC), and various 3rd party services, would benefit from full and direct access to the power of the backend (read: faster, more complete, more API-stable) unAPI (um) API. Related, many moons ago, I built a perl module (OpenILS::Utils::TagURI) to make parsing of tag URIs systematic for use in OpenSearch and unAPI contexts. This branch uses O::U::TagURI and a simple shim inside open-ils.supercat to expose the power of that backend unAPI functionality.

Code coming soon. Watch this space...

[Mike Rylander (mrylander)]

Branch, as promised:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp_1367926-extended-unapi

[Kathy Lussier (klussier)]

I took a look at this code on a MassLNC Sandbox this morning. I was able to retrieve data using the copy's barcode and played around with adjusting which bits of data should be retrieved.

However, I had trouble retrieving output by record ID.

Using the example MIke included in his git branch, I tried the following URL:
http://mlnc2.mvlcstaff.org/opac/extras/unapi?tag::U2@bre/167{holdings_xml,acn,acp,mra}&format=mods32

I get then get the following message: Sorry, we couldn't a record with the id of in format mods32.

This message appears with any format I try to use.

I ultimately was able to successfully retrieve output using the following URL (note the addition of id= after the question mark):
http://mlnc2.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/167{holdings_xml,acn,acp,mra}&format=marcxml

However, marcxml is the only format that works with the above URL. If I try mods32, which was the format used in Mike's example, I get an XML parsing error:not well formed.

Also noting that this branch needs a release notes entry, but I can add one when I sign off on it. I just want to verify what the correct URL format should be for the bre example and if the mods32 format (or others) should be working here.

Thanks Mike!
Kathy

[Josh Stompro (u-launchpad-stompro-org)]

I don't think this is quite ready, the fact that "format=" should show all the possible formats, but the only format that seems to work is marcxml should be addressed.

unapi?id=tag::U2@acp/FRE410001102{holdings_xml,acn,acp,mra,bre}/-/0/barcode&format=html
returns marcxml

unapi?id=tag::U2@bre/165&format=mods32
XML Parsing Error: not well-formed

The full format looks to be
http://example.com/opac/extras/unapi?id=<classname>/<id>[<paging>]{<includes>}/<location>/<depth>/<pathinfo>?format=

I tried to figure out why only marcxml seemed to be a supported format, but I couldn't figure out where the issue was.

I'm curious why the regex is set to match /-/0/barcode as the pathinfo, I'm assuming it is just a shortcut to leaving location and depth unset if location = "-"?

Josh

[Mike Rylander (mrylander)]

Josh,

For U2 tag URIs, only the bre object type supports differing formats, and the rest expect to have a format of xml. Anything else is for the future. That's a limitation of the underlying unapi DB functions. The format list is an artifact of the version 1 implementation, but we can probably arrange to trim it in the face of a version 2 tag URI for the time being. As for the regex, yes,

Kathy and Josh,

Can you tell where the XML parsing error is coming from? Is it just a browser complaint? Do you still have the branch loaded somewhere that I could use wget or curl to pull from, or, can you pull the result and share it?

I'll work up some release notes soon.

Thanks!

[Mike Rylander (mrylander)]

er ... hit send too soon.

... As for the regex, yes, location="-" and depth="0" is the default, meaning "everywhere". "-" is the canonical stand-in for "top of the org tree" in supercat.

[Josh Stompro (u-launchpad-stompro-org)]

RE: XML Parsing Error, in my case it is from Firefox, it just doesn't like the xml, but the content is there. Here is a pastebin of the output that Firefox doesn't like. My test server is still up but isn't accessible externally.

Using the sample DB load data.
/opac/extras/unapi?id=tag::U2@bre/165&format=mods32
http://pastebin.com/N6pa9d7q

Josh

[Mike Rylander (mrylander)]

Josh,

xmllint doesn't seem to mind that content, and it's definitely is mods. I think FF is just complaining about the embedded (possibly broken or otherwise non-UTF8 encoded) non-ascii chars. That, of course, would be a data of FF problem.

Thanks for grabbing the output!

[Mike Rylander (mrylander)]

I've rebased the branch to master and added the ability to include non-MARC fields from biblio.record_entry. New branch is here:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp_1367926-extended-unapi-rebase-bre_extern

[Galen Charlton (gmc)]

I've reviewed and squashed Mike's patches, and the branch collab/gmcharlt/lp1367926-unapi2 in the working/Evergreen repository is now ready for review:

  http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/gmcharlt/lp1367926-unapi2

This branch also addresses the following concerns:

* lack of release notes
* character encoding
* the list of available formats (when given a U2 ID) now includes only the applicable ones

It also adds some unit tests.

[Kathy Lussier (klussier)]

Thanks for the new branch Galen! This looks good.

I have a few questions:

1. For the release notes, where you explain how to form the URL, should we include something that says &format=xxxx should be tacked on to the end of the URL? I can make the changes to the release notes, but I wasn't sure if it really was needed since I'm unclear on whether the format= is necessary. I do see that, if it's missing, the available formats are returned as you described in your above comment.

2. I'm not sure if this is a bug or a problem with my URL. I was trying some of the different class names and encountered problems when I tried cbs. Using this URL:

http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@cbs/3&format=mods32

I get a bib source of Project Gutenberg instead of the correct bib source, oclc. When I use the class in an include, it properly lists the bib source as OCLC. See

http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/3{holdings_xml,acn,acp,mra,cbs}&format=mods32

Is this a bug or user error?

[Galen Charlton (gmc)]

> 1. For the release notes, where you explain how to form the URL, should
> we include something that says &format=xxxx should be tacked on to the
> end of the URL? I can make the changes to the release notes, but I
> wasn't sure if it really was needed since I'm unclear on whether the
> format= is necessary. I do see that, if it's missing, the available
> formats are returned as you described in your above comment.

Yeah, a note about the format parameter would be good to have in the
release notes.

> 2. I'm not sure if this is a bug or a problem with my URL. I was trying
> some of the different class names and encountered problems when I tried
> cbs. Using this URL:
>
> http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@cbs/3&format=mods32
>
> I get a bib source of Project Gutenberg instead of the correct bib
> source, oclc. When I use the class in an include, it properly lists the
> bib source as OCLC. See
>
> http://mlnc1.mvlcstaff.org/opac/extras/unapi?id=tag::U2@bre/3{holdings_xml,acn,acp,mra,cbs}&format=mods32
>
> Is this a bug or user error?

An opportunity for documentation clarification. :)

"tag::U2@cbs/3" signifies that you want *just* the config.bib_source
row whose ID is 3.

[Kathy Lussier (klussier)]

I added a commit to the release notes to add the format to the URL and merged it to master.

Thanks so much for your work on this Mike and Galen!