Strip down, simplify, and de-bug-ify Evergreen EDI parsing

Bug #1065270 reported by Bill Erickson
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Evergreen
Fix Released
Undecided
Unassigned
2.3
Fix Released
Undecided
Unassigned

Bug Description

EDI for libraries is simply a shared mechanism for packaging up a few pieces of ACQ order information. We don't need the full might of EDI/edi4r for interpreting vendor EDI, nor do we need to validate messages from vendors. We only need to extract a handful of data and get on with it.

The EDI stack in Evergreen is complicated. It involves multiple external Ruby libraries (edi4r, openils-mapper), a Ruby web gateway (since Evergreen isn't written in Ruby), Business::EDI, and a pile of Evergreen Perl code to perform additional extractions on the data returned from the Ruby code. Apart from requiring many layers, the code creates a scenario where we translate EDI into JEDI, which itself contains EDI meta-codes, which are part of the spec, but are not always present when visually inspecting an EDI message. In other words, you have to know a lot about EDI to use the final data structure and you have to refer to the EDI and the spec to decipher the data structure. And, of course, determining where in the chain a problem has occurred is complicated by the various, multi-language layers.

This code replaces all of that (for inbound EDI) with a single Perl mod that extracts the pieces we need directly from the raw EDI in a format that's simple to use/read. The new code is not an EDI library. It extracts the needed data via regular expressions based on the rules defined by http://www.editeur.org/31/Library-Book-Supply/ (and tested w/ vendor data) and produces a list of messages w/ a minimal amount (primarily for backwards compat) of raw EDI information. Along the way, it fixes a number of bugs.

Some are referenced in bug #1052647

Other issues/improvements:

* Treat each message body (i.e. each ORDRSP or INVOIC) as its own edi_message
  instead of storing the collection as a whole in one edi_message.
* Improved debug and informational logging
* Reduces xact begin/commit's

======

working => user/berick/edireader

Changed in evergreen:
milestone: 2.3.1 → 2.4.0-alpha
Revision history for this message
Bill Erickson (berick) wrote :

Merged in master and pushed another commit for checking to make sure we don't try to process nonexistent POs in order responses.

Revision history for this message
Jason Stephenson (jstephenson) wrote :

Should this bug still affect 2.3? Is the code that was merged going to be backported?

Changed in evergreen:
status: New → Fix Committed
Revision history for this message
Ben Shum (bshum) wrote :

I think berick meant he merged master into his working branch. Not that it's in master (or at least I can't seem to find the related EDI commits in the listed branch in master yet).

Changed in evergreen:
status: Fix Committed → Triaged
Revision history for this message
Bill Erickson (berick) wrote :

What Ben said. Apologies for the confusing language. The code remains un-merged.

Revision history for this message
Bill Erickson (berick) wrote :

Pushed release notes commit to working branch.

Revision history for this message
Lebbeous Fogle-Weekley (lebbeous) wrote :

This is code that has seen testing on a server provided by the SC Lends consortium with participation by staff from MassLNC and Sitka. It is about to get one more final pre-merge bit of testing from me

Changed in evergreen:
status: Triaged → In Progress
assignee: nobody → Lebbeous Fogle-Weekley (lebbeous)
Revision history for this message
Lebbeous Fogle-Weekley (lebbeous) wrote :

And pushed to master and rel_2_3.

Changed in evergreen:
status: In Progress → Fix Committed
assignee: Lebbeous Fogle-Weekley (lebbeous) → nobody
Ben Shum (bshum)
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.