SIPServer is heavy, not my brother

[Mike Rylander]

SIPServer is inordinately heavy, memory-wise, at least for Evergreen. This is primarily because connections are stateful and the ILS driver for Evergreen generates a lot of code. In-memory footprint of 65MB+ is common for a quiescent server, and larger for active sites. Add to that the fact that active sites have more SIPServer processes running, and you've got a lot of wasted RAM.

SIPServer is based on Net::Server, and Net::Server has a server personality called Multiplex, which allows a small number of backends to process many requests. However, because of the statefulness, this requires maintaining some state must be maintained, and passed, to the stateless backends. The following branch does that, but requires OpenSRF 2.3+ to be safe in production.

Use of the multiplex personality is optional based on a setting in the sip config file.

http://git.evergreen-ils.org/?p=working/SIPServer.git;a=shortlog;h=refs/heads/collab/berick/sip2-support-multiplex-personality

[Galen Charlton (gmc)]

The Evergreen side of this is available in the collab/gmcharlt/eg-sip2-support-multiplex-personality branch of the working/Evergreen repository:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/gmcharlt/eg-sip2-support-multiplex-personality

Note that the patches are dependent on those for bug 134871

[Mike Rylander (mrylander)]

Putting the pullrequest tag back on here now that the EG-side stuff is LP'd. Thanks, Galen, for that.

[Mike Rylander (mrylander)]

A "3" got dropped in the above-linked bug. It's actually 1348731 for the Evergreen-side stuff.

[Mike Rylander (mrylander)]

Thanks, LP, for not linking that bug.

https://bugs.launchpad.net/evergreen/+bug/1348731

[Mike Rylander (mrylander)]

I've pushed the Evergreen bits supporting this to master, 2.6 and 2.5.

[Bill Erickson (berick)]

Lightly squashed branch w/ a number of multiplex additions and repairs pushed to:

http://git.evergreen-ils.org/?p=working/SIPServer.git;a=shortlog;h=refs/heads/collab/berick/sip2-support-multiplex-personality-squash1

It contains sign-off's for all of the commits which are not mine.

Beware it also contains the two commits from bug #1338761 (Relais extensions), since they were tested together and extracting them was getting messy. I suggest merging the Relias work before (or along with) this code.

[Mike Rylander (mrylander)]

And now here's a heavily squashed version. Changes/updates from before include lots of bug fixing and forked-worker keepalive for high-rate clients to reduce latency.

This is in production now and working well. The main resource reduction due to this branch is a drop from essentially unbounded memory (due to leaking sessions, and their size), to ~3G total system RAM use with 75 connections, including 15-20 high-rate clients.

http://git.evergreen-ils.org/?p=working/SIPServer.git;a=shortlog;h=refs/heads/user/miker/timeout-keepalive-relais-status_before_login-multiplex-final_squash

[Ben Shum (bshum)]

Marking review target to 2.next, sounds promising!

[Ben Shum (bshum)]

Or wait, is this for SIPServer? Hmm... (not sure on targets now)

[Blake GH (bmagic)]

We put this patch into production this week and we have been very pleased. The number of running SIPServer processes doesn't climb out of control!

Sign off:

http://git.evergreen-ils.org/?p=working/SIPServer.git;a=commit;h=b0786d3ae13fedb87b13463c18e1bfb43df9c3b5

( had to do some editing to correct some merge conflicts against master )

[Mike Rylander (mrylander)]

Blake,

Thanks, though the rebase seems unnecessary -- user/miker/timeout-keepalive-relais-status_before_login-multiplex-final_squash is already rebased to master. I'll add your signoff to the final commit and push soon.

Thanks!

[Mike Rylander (mrylander)]

And, it's been pushed. Thanks!

[Ben Shum (bshum)]

Changing milestone to reflect real target of 2.8 series - 2.8 beta, here we come!