Evergreen 2.10
I set out to make various changes to the hold targeter, but ended up rewriting it, because it allowed me to do some heavy refactoring. Linked is branch with the new code. It adds some features, improves targeting speed, and refactors the code in a way that is hopefully easier to understand and expand. (The existing targeter code has acquired a lot of additions over the years. It was probably time for general refactoring and optimization).
The goal was to mimic the existing targeter logic, with the exception of some changes to the max hold target loop behavior.
I've tested the code on 3 data sets. On 2 of them, I have confirmed that the final hold copy maps are identical and that the number of holds with non-NULL current copies match. (The exact targeted copies will vary due to a certain amount of randomness in the targeting logic).
On the relatively small concerto data set, I get a 10% increase in batch targeting speed. On a larger data set of 515 holds (w/ local hold configs), it runs 6 times faster overall. On an even larger data set of 280k holds (in a multi-server environment), it runs 4.5 times faster as a single process than the traditional hold targeter running with 3 parallel.
These speed improvements are, of course, highly subjective and will be affected by local configuration and system resources. It's encouraging, though, that it performs faster in every scenario I've tried so far (assuming the speed improvement are the result of efficiency and not logic errors).
In all cases above, it uses significantly less RAM, because open-ils.storage is not used. DB communication uses cstore.
Changing a complex, core part of the ILS logic is no small thing. This will require lots of additional testing and review.
I'll document features and changes in a follow-up comment.
Thoughts, questions, feedback, and testing appreciated.
http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/hold-targeter-reify-expand
From the features documented in the commit:
* Ports hold targeter code to a Perl utility function, communicating w/ the DB via cstore instead of storage.
* Adds a new global flag 'circ.holds.
* Adds a new DB function to regenerating hold copy maps to make map deletion and creation more efficient.
* Adds an option for targeting holds in newest to oldest order.
* Caches all org unit settings per targeter run.
* Adds support for "skip_viable" option. This tells the hold targeter to avoid modifying any holds that target viable copies. AKA "fix broken" mode.
For example, you might run in skip_viable mode with a retarget interval of 24hr once a day to repair non-viable holds, then also run the targeter in regular mode once a day with a retarget interval of 48 hours to give staff 2 days to process viable holds.
* Hold target loops logic changes:
** Org units with fewer target attempts are prioritized during loop processing. So, instead of segregating org units into 2 categories, those attempted in the current loop and those not attempted, sort org units by the number number of times they have been attempted. Within each grouping, prioritize by target weight/proximity as before.
** Target looping treats the pickup lib like any other org unit. If a targeted copy at the pickup lib remains un-captured, at re-target time, a copy at a different branch is chosen (if one is available) even if other copies at the pickup lib are targetable.