Server-generated print templates

[Bill Erickson]

Evergreen 3.3

Continuing from email thread:

http://libmail.georgialibraries.org/pipermail/open-ils-dev/2019-April/010709.html

Recap:

This is a proposal to migrate the process of print template compilation / generation away from the browser (or any other client) to the server. Templates will be implemented with Template Toolkit. At print time, instead of having the client fetch the template and process it locally, the client posts the print data to the server, which generates the selected template with the provided data, returning the final product as a print-ready chunk of html, text, etc.

Key benefits:

1. A print template language / process which persists across (and is agnostic of) any client technology. This is especially important given the requirements for compiling Angular templates.

2. Centrally managed, per-org unit print templates.

3. Configurable template locales and content types (html, text, etc.)

4. Options for added template security (strict edit permissions, configurable HTML scrubbing).

Proposed Migration Plan:

1. Server print templates will have an "active" flag to prevent a template from being used if it's under construction.

2. AngJS will first try to load a saved template, followed by the stock template file (staff/share/print_templates/*) as before, if it finds neither, it will attempt to process the template on the server.

3. As templates are added to the seed data, their matching staff/share/print_templates/* file will be deleted along w/ instructions to admins to remove their copy of the file and matching workstation settings once they have reviewed the server-hosted version of the template.

4. On the Angular side, the print caller either passes an Angular template reference, indicating a locally compiled template, or the name/owner/etc of the server-hosted template for remote compilation.

5. Browser clients will retain the ability to process templates locally for special circumstances, like offline interface receipts.

==

The WIP branch I posted to the list:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/server-print-templates

Will clean this up and repost w/ release notes, etc.

[Bill Erickson (berick)]

Cleaned up branch pushed:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1825851-server-generated-print-templates

Paraphrasing part of the release notes:

To me, the main benefit of this feature is we have a way to manage print templates for Angular interfaces. Print templates for AngJS interfaces will have to be migrated to server-managed as each UI is ported to Angular. Whether we want to start migrating AngJS print templates before the interfaces are migrated is an open question.

[Bill Erickson (berick)]

The branch also includes a sample configuration using HTML::Restrict to scrub HTML content for security. The code is commented out, though, since it requires a new cpan-installed Perl module. I wasn't quite ready to make that a requirement for testing and/or whether we wanted to consider other alternatives.

[Galen Charlton (gmc)]

I have a qualm about the proposed migration path, particularly the bit about admins removing files from staff/share/print_templates.

An industrious admin who finishes converting their templates ahead of the seed data would have to take additional steps if they apply maintenance releases to prevent the older templates from coming back. An industrious user without server access would require assistance from a server admin to finish migrating a template. Not necessarily huge deals, but they do add friction.

As an alternative, would it be possible to have server-side printing be tried first (maybe in conjunction with adding a method to get a list of available server-side templates that gets invoked during app startup)?

It looks like HTML::Defang is packaged for Debian and Ubuntu; would that serve for HTML cleanup?

[Bill Erickson (berick)]

All good suggestions. Thanks, Galen. Looking into these now.

[Bill Erickson (berick)]

Force-pushed back to same branch implementing Galen's suggestions.

Release notes updated to reflect changes.

For HTML::Defang, the default configuration seems sane so I opted not to tweak it.

[Bill Erickson (berick)]

Updated branch pushed with loads of changes:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1825851-server-generated-print-templates-2

* Adds template helpers for date formatting (etc.)

* Adds additional data to the template environment representing information about the caller (time zone, org unit, staff info, etc.).

* Adds an implementation of the "Holds for Bib Record" template, called from the Angular staff catalog Holds tab. Template is based on wide_hash objects.

* Adds IdlService toHash() function.

* Adds combobox option to allow the caller to override the default display template, useful for appending (e.g.) the org unit shortname to a list of values that would otherwise be duplicated in the dropdown.

* Tweaks CStoreEditor so that it's possible to apply a personality (e.g. open-ils.pcrud) to a specific instance without applying the change process-globally. This fixes an issue where using the pcrud personality in a mod_perl handler affected all editor instances in the process.

* Adds template clone and delete operations to the UI.

Finally, I have removed all of the AngularJS additions from my previous branch. This may be controversial. The key use case for this feature is to support printing in Angular. As each Angular interface is migrated from AngJS, part of the process will be to migrate any related print templates to server templates.

While I can imagine a use case for porting receipts before a UI is migrated, it adds a bit of complication, so I've taken it out for now. The code lives on in the original branch, of course, so I can be easily recovered if we want to retain the functionality. Feedback always appreciated.

[Kyle Huckins (khuckins)]

I've made the changes to eg_vhost and eg_startup, and installed the Perl dependency, as noted in the release notes, and noticed a few things while testing this.

When I attempting to print holds to test the holds_for_bib print template, I receive a POST 404 to the /print_template, "Error compiling template" and "Uncaught (in promise): Error: Error compiling server-hosted print template," and the attempt fails. This was due to the template not being active by default. Some form of error-handling feedback may be useful here, such as a toast saying something to the effect of "No valid print template found."

In the Print Templates Admin UI, when cloning a template, the new template is partially selected(the editor on the left seems to display the correct information, but everything on the right side is missing. The template dropdown is cleared, and selecting the new clone has no effect unless you select another already-existing template first. This will bring the compiled info and preview back, though it will always represent the template you selected prior to selecting the clone. When cloning, even if "active" is selected, it will display as inactive until you edit the clone to be active.

When switching templates while on the sample data tab, a console error appears, "TypeError: Cannot set property 'innerHTML' of null."

[Bill Erickson (berick)]

Thanks, Kyle! Rebasing to master and looking at some of these issues...

[Bill Erickson (berick)]

Rebased continuation branch pushed:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1825851-server-generated-print-templates-3

From the commits:

==

1. When cloning a template, be sure the cloned template is available in the template selector regardless of whether it would have been given the current filters.

2. When cloning, set the 'active' flag explicitly to false so the user is forced to manually activate. This helps prevent unintentional activation on in-process templates.

3. When cloning, clear the owner value so the user is forced to select an owner value.

4. Allow the template editor textarea to expand vertically as the template gets longer (i.e. adds more new lines).

==

Display error toasts when an attempt is made to a server-generated print template and no active template can be found or the template generation failed.

This required moving String and Toast components/services into the base module so they could be used by the print components/services.

==

[Kyle Huckins (khuckins)]

Thanks Bill, that resolves the issues I noted. I did notice one new thing - it's possible to hit save when the owner field is empty, which closes the modal and leaves an error in the console that the print template failed to create. Failing to be created is expected, but we may want to have the owner field be required, in the same way that Name and Label are, graying out the Save button if unselected

[Bill Erickson (berick)]

Thanks, Kyle. I confirmed the 'owner' field is marked as required in the IDL, so this is more of an FMEditor / OrgSelect bug. I'll open a new LP for that.

[Kyle Huckins (khuckins)]

With that in mind, I have tested this code and consent to signing off on it with my name, Kyle Huckins, and my email address, <email address hidden>.

[Bill Erickson (berick)]

Opened bug #1838784 for the org-select / fieldmapper editor form validation issue.

[Galen Charlton (gmc)]

Pushed to master for inclusion in 3.4. Thanks, Bill and Kyle!