fine generator can leak memory when run often

Bug #1042924 reported by Bill Erickson
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Fix Released
Medium
Unassigned
2.3
Fix Released
Medium
Unassigned
2.4
Fix Released
Medium
Unassigned

Bug Description

Evergreen 2.1+

From the commit:

Calling "next" from within a "try" block results in a memory leak, presumably because "try" is a tangled nest of subs and evals. Replacing the "try" with a good ol' "eval" avoids the leak.

This can be reproduced with the following:

---------
use Error qw/:try/;

foreach (0..200000) {
    try {
        next;
    } catch Error with {
    };
}
---------

This particular leak in the fine generator is onerous when the fine generator is run often (e.g. every 15 mins), which means circs that have already been processed for the day are re-analzyed over and over, causing the code to continue early (next) to the next loop iteration for large numbers of circs. It also happens when a circs are skipped because they have no fine interval, rate, or max fine.

You know this is happening because you will see something like this in the storage stderr log:

Exiting eval via next at
/usr/local/share/perl/5.10.1/OpenILS/Application/Storage/Publisher/action.pm
line 820.

Exiting subroutine via next at
/usr/local/share/perl/5.10.1/OpenILS/Application/Storage/Publisher/action.pm
line 820.

This patch does not avoid the "exiting eval via next" warning, since we're still next'ing out of the eval. It just avoids the memory leak (and the "Exiting subroutine" warning). More extensive refactoring is needed to to completely remove the second warning.

=========

With the right combination of factors (running the fine gen often, re-using the same storage drones over and over to process fines), open-ils.storage processes can grow to consume Gigs of memory in a matter of days, prompting manual service restarts.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/fine-gen-try-mem-leak

Tags: pullrequest
Changed in evergreen:
status: New → Triaged
Ben Shum (bshum)
Changed in evergreen:
milestone: none → 2.4.0-rc
importance: Undecided → Medium
Ben Shum (bshum)
Changed in evergreen:
milestone: 2.4.0-rc → none
Ben Shum (bshum)
Changed in evergreen:
milestone: none → 2.5.0-m1
Dan Wells (dbw2)
Changed in evergreen:
milestone: 2.5.0-m1 → 2.5.0-m2
Revision history for this message
Mike Rylander (mrylander) wrote :

Let's stop kicking this leaky can down the road. Thanks, Bill!

Changed in evergreen:
status: Triaged → Fix Committed
Revision history for this message
Ben Shum (bshum) wrote :

Marking fix released for 2.3 and 2.4 since the commit was done before the 2.3.9 and 2.4.1 releases.

no longer affects: evergreen/2.2
Ben Shum (bshum)
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.