open-ils.hold-targeter drones consistently use a lot of memory

[Jason Stephenson]

Evergreen 3.2.10
OpenSRF 3.1.1
PostgreSQL 9.6.18

We have found that the open-ils.hold-targeter drones consistently use a lot of memory on our utility server that runs the new hold targeter (see attached screenshot). In one recent instance, it nearly filled the swap partition from using so much RAM. We have to occasionally restart the service to prevent this.

Furthermore, they seem to run a long time. We are using the stock configuration for number of connections, etc.:

      <open-ils.hold-targeter>
        <keepalive>3</keepalive>
        <stateless>1</stateless>
        <language>perl</language>
        <implementation>OpenILS::Application::HoldTargeter</implementation>
        <max_requests>17</max_requests>
        <unix_config>
          <unix_sock>open-ils.hold-targeter_unix.sock</unix_sock>
          <unix_pid>open-ils.hold-targeter_unix.pid</unix_pid>
          <max_requests>1000</max_requests>
          <unix_log>open-ils.hold-targeter_unix.log</unix_log>
          <min_children>1</min_children>
          <max_children>15</max_children>
          <min_spare_children>1</min_spare_children>
          <max_spare_children>5</max_spare_children>
        </unix_config>
        <app_settings>
        </app_settings>
      </open-ils.hold-targeter>

[Bill Erickson (berick)]

Dropping <max_requests>1000</max_requests> in the default EG config will be a start. I run with max_requests=10 on a dedicated utility server. That may be too aggressive as a default, though.

Another thing that would help is a parameter option for the hold targeter that tells it to recycle itself once it's done processing large batches.

We can do that with OpenSRF's force drone recycle option:

https://git.evergreen-ils.org/?p=OpenSRF.git;a=commitdiff;h=304365165e7ba0cc08bb6c5f0ba25f0b541fd27d

[Jason Stephenson (jstephenson)]

Bill, thanks for the configuration suggestion. I will try that starting next week. We're going to install some patches this Sunday (7/26/20).