Web client: Barcode file upload inflates pcrud drone count

Bug #1745499 reported by Jeff Davis on 2018-01-25
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Evergreen
Medium
Unassigned
3.0
Medium
Unassigned

Bug Description

EG 3.0.2

When uploading a file containing patron or item barcodes, the web client loops through the uploaded file line by line and fires off nearly-simultaneous pcrud requests for each barcode. This tends to inflate the number of pcrud child processes.

Patron barcodes:
1. Go to Circulation > User Buckets > Pending Users.
2. Click "Choose file" and upload a file containing a large number of patron barcodes.

Item barcodes:
1. Go to Circulation > Item Status.
2. Click "Choose file" and upload a file containing a large number of item barcodes.

(I don't see a file upload option for record or copy buckets in 3.0.2.)

The default configuration for open-ils.pcrud has max_children = 15. With this configuration, uploading a file containing 30 or more barcodes will quickly spawn the maximum allowable number of pcrud drones. Increasing pcrud max_children mitigates the issue somewhat but still results in an inflated drone count: for example, with max_children = 100 and uploading 100 barcodes at a time, a single upload spawned 22 separate pcrud drones.

I'm not sure if there is any inherent problem in having a large number of pcrud drones. My gut feeling is that it would be good practice not to use so many separate processes for batch barcode retrieval.

Bill Erickson (berick) on 2018-01-25
Changed in evergreen:
status: New → Confirmed
assignee: nobody → Bill Erickson (berick)
Bill Erickson (berick) wrote :

Fix for the patron barcode lookup pushed:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1745499-file-upload-pcrud-blast

I have not looked at the item status UI yet. Removing myself as assignee for now.

Changed in evergreen:
assignee: Bill Erickson (berick) → nobody
Bill Erickson (berick) on 2018-01-26
Changed in evergreen:
assignee: nobody → Bill Erickson (berick)
Bill Erickson (berick) wrote :

Fix for item status UI pushed to same branch. This code could not be batched in the same was as the patron barcode lookup code, because it does a more than just issue a pcrud search. I opted to serialize the lookup instead and, since this is one of those rare cases where we know how many of something we are fetching, I added a progress dialog that indicates fetch progress in real time.

tags: added: pullrequest webstaffclient
Changed in evergreen:
milestone: none → 3.0.4
assignee: Bill Erickson (berick) → nobody
Galen Charlton (gmc) wrote :

Looks good to me. Signoff branch here: user/gmcharlt/lp1745499_signoff

tags: added: signedoff
Changed in evergreen:
importance: Undecided → Medium
Mike Rylander (mrylander) wrote :

I suspect that copy buckets may exhibit this behavior as well. Looking now...

Mike Rylander (mrylander) wrote :

My suspicion was based on a false memory of copy buckets have barcode upload. They don't (yet). Nothing to see here, move along...

Changed in evergreen:
assignee: nobody → Jason Stephenson (jstephenson)
Jason Stephenson (jstephenson) wrote :

Works for me, also! Pushed to master and rel_3_0.

Thanks, Bill, Galen, and Jeff!

Changed in evergreen:
milestone: 3.0.4 → 3.1-beta
assignee: Jason Stephenson (jstephenson) → nobody
status: Confirmed → Fix Committed
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers