find_org and get_org_tree utility methods are expensive and slow down every search in a large installation

[Mike Rylander]

While looking at container UI timeouts, we discovered that, for large org trees, some several seconds are spent testing org visibility. The immediate cause is that AppUtils::get_org_tree() does not populate the process-local cache with a memcache-cached org tree. That only happens when memcache does not have a copy of the org tree. This is obviously a simple oversight.

Additionally, the visibility check is making heavy use of lots of indirection and delegation to utility code, when some slightly smarter code could avoid many repeated function calls.

Branch forthcoming to address both issues.

[Mike Rylander (mrylander)]

Branch up at: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp-1836963-get_org_tree-and-find_org-search-optimization

[Galen Charlton (gmc)]

Tested and works for me. Signed-off pushed to user/gmcharlt/lp1836963_signoff.

I've attached a test script that can be used to check timings before and after applying the patch. To create enough OUs in a Concerto database to make the performance increase apparent, you can run something like this:

insert into actor.org_unit (parent_ou, ou_type, ill_address, holds_address, mailing_address, billing_address, shortname, name, opac_visible)
select distinct 6, 4, 9, 9, 9, 9, 'BR3SL-' || upper(first_given_name || family_name), first_given_name || ' ' || family_name || ' Sub-library', true
from actor.usr;

[Galen Charlton (gmc)]

As a side note, I suspect that the reason why fixing how the per-process cached used by $U->get_org_tree() matters so much is the expense of JSON-to-Perl deserialization in OpenSRF::Utils::Cache->get_cache(), which adds up for processing each OU in a large OU tree.

[Galen Charlton (gmc)]

We have been using this in production for a couple months, and given the time that has passed, I've pushed this to rel_3_4, rel_3_3, rel_3_2, and rel_3_1. Thanks, Mike!