find_org and get_org_tree utility methods are expensive and slow down every search in a large installation

Bug #1836963 reported by Mike Rylander
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Medium
Unassigned
3.1
Medium
Unassigned
3.2
Medium
Unassigned
3.4
Undecided
Unassigned

Bug Description

While looking at container UI timeouts, we discovered that, for large org trees, some several seconds are spent testing org visibility. The immediate cause is that AppUtils::get_org_tree() does not populate the process-local cache with a memcache-cached org tree. That only happens when memcache does not have a copy of the org tree. This is obviously a simple oversight.

Additionally, the visibility check is making heavy use of lots of indirection and delegation to utility code, when some slightly smarter code could avoid many repeated function calls.

Branch forthcoming to address both issues.

Revision history for this message
Mike Rylander (mrylander) wrote :
tags: added: pullrequest search
Galen Charlton (gmc)
Changed in evergreen:
milestone: none → 3.3.3
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Galen Charlton (gmc)
Revision history for this message
Galen Charlton (gmc) wrote :

Tested and works for me. Signed-off pushed to user/gmcharlt/lp1836963_signoff.

I've attached a test script that can be used to check timings before and after applying the patch. To create enough OUs in a Concerto database to make the performance increase apparent, you can run something like this:

insert into actor.org_unit (parent_ou, ou_type, ill_address, holds_address, mailing_address, billing_address, shortname, name, opac_visible)
select distinct 6, 4, 9, 9, 9, 9, 'BR3SL-' || upper(first_given_name || family_name), first_given_name || ' ' || family_name || ' Sub-library', true
from actor.usr;

tags: added: signedoff
Changed in evergreen:
assignee: Galen Charlton (gmc) → nobody
Revision history for this message
Galen Charlton (gmc) wrote :

As a side note, I suspect that the reason why fixing how the per-process cached used by $U->get_org_tree() matters so much is the expense of JSON-to-Perl deserialization in OpenSRF::Utils::Cache->get_cache(), which adds up for processing each OU in a large OU tree.

Changed in evergreen:
milestone: 3.3.3 → 3.3.4
Changed in evergreen:
milestone: 3.3.4 → 3.3.5
Revision history for this message
Galen Charlton (gmc) wrote :

We have been using this in production for a couple months, and given the time that has passed, I've pushed this to rel_3_4, rel_3_3, rel_3_2, and rel_3_1. Thanks, Mike!

Changed in evergreen:
status: Confirmed → Fix Committed
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers