EDI: fetcher's check of duplicate files can be too slow

Bug #1673799 reported by Galen Charlton
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Evergreen
Fix Released
Medium
Unassigned
2.11
Fix Released
Medium
Unassigned
3.0
Fix Released
Medium
Unassigned

Bug Description

When acq.edi_message is very large (either because of the passage of time or as a consequence of bug 1662902), the query that edi_fetcher.pl runs to see if a remote file had been previously proceeded can take a long time. In particular, we've observed a case where it could take over two minutes, thereby running into a timeout and causing edi_fetcher.pl to crash.

The query in question currently uses "ilike", for which PostgreSQL does not well support use of indexes; I propose to replace that with use of evergreen.lowercase().

Evergreen master

Galen Charlton (gmc)
tags: added: acq edi performance
Changed in evergreen:
importance: Undecided → Medium
milestone: none → 2.12.1
Revision history for this message
Galen Charlton (gmc) wrote :

A patch is available in the user/gmcharlt/lp1673799_speed_up_dupe_edi_file_check branch in the working/Evergreen repository:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmcharlt/lp1673799_speed_up_dupe_edi_file_check

tags: added: pullrequest
Changed in evergreen:
milestone: 2.12.1 → 2.12.2
no longer affects: evergreen/2.10
Bill Erickson (berick)
Changed in evergreen:
assignee: nobody → Bill Erickson (berick)
Revision history for this message
Bill Erickson (berick) wrote :

Confirmed modified query and index resulted in queries taking milliseconds instead of seconds. Also confirmed new query is looking for the same data. Merged to 2.11 and higher. Thanks, Galen!

Changed in evergreen:
status: New → Fix Committed
assignee: Bill Erickson (berick) → nobody
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.