Programmatic access to report output

[Jeff Davis]

Suppose there is a power user at your library who is writing a program to display monthly circ stats. This user does not have direct access to the Evergreen database. The reporter is the right tool for gathering the data, but obtaining the output is a manual process: you wait for an email, click on a link that was generated at runtime, and click on another link to download your report output in the desired format. It would be nice if the report output could be made available at a consistent, predictable URL, so that the power user's program can download it easily and automatically.

[Terran McCanna (tmccanna)]

+1

If the process generated a file with a predictable name, that file could then be accessed by scripts without manual intervention every time it got updated - "new books" sliders come to mind as an example.

[Lynn Floyd (lfloyd)]

+1
I would love not have to go and download each report each time.

[James Fournie (jfournie)]

+1 I'm Jeff's "power user". The way our old ILS did this was that you could specify a target S/FTP location to send the report output. It's not ideal, an API would be better, but I wonder if maybe the SendFile action trigger could do this somehow with minimal additional work?

[Rosie Le Faive (rlefaive)]

Hi! This is my use case! I want to be able to run regular stats and checks to make sure our giant batch loads/updates of e-records (at MPOW a 1FTE job) don't go awry. Being able to access machine readable report outputs from EG directly would be awesome. (that said, i'd be interested in hearing about custom scripts/tools that do this)

[Jessica Woolford (jwoolford)]

This came up as a need at the Reports Interest Group meeting at the 2021 conference. Attendees expressed the desire for the results to be JSON or CSV.

[Josh Stompro (u-launchpad-stompro-org)]

I wonder if this could be taken care of with some apache rewriting magic, or something of that nature.
Just thinking out loud about this problem.

Path to a report output looks like "reporter/161/160/647" "reporter/template/report/run"

So maybe we can rewrite "reporter/161/160/newest" to point to the most recent run for a template/report just by picking the largest index number in that folder?

So reporter/161/160/newest/report-data.html.raw.html would always give the most recent output for a certain report? CSV is an option already, so point to the CSV file if you want. If JSON output gets added we could use that instead.

A simple way to do it -
An apache rewrite map program could be used to return the largest folder given a template and report number.
https://httpd.apache.org/docs/2.4/rewrite/rewritemap.html#prg

Is it safe to assume that the largest sequential ID number is the newest? I'm aware that rolled back transactions could skip ID numbers, but I don't think those are re-used until the sequence rolls over. If we are using a program to look up the ID Numbers, maybe it makes sense to just to look up the most recent run in the DB though.

Josh