Missing close() after shutdown() in EGCatloader/Record.pm FD Leak

Bug #1478123 reported by Josh Stompro
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released
Fix Released
Fix Released

Bug Description

Running EG 2.8.2, OpenSRF 2.4.1, Debian Jessie, OpenVZ Container on a Proxmox VE 3.4 host.

See OpenILS-General list posts about this issue as I worked through trying to figure out what the issue was.

The Summary is that in the EGCatLoader/Record.pm there are some functions that make requests to load added content so it can later be used by the template toolkit, in those functions the http requests are shutdown to get rid of them, but they are not closed also. When a socket is just shutdown the file descriptor isn't destoyed, so it just sticks around until apache is restarted.

I noticed this because OpenVZ has various resource limits, one of them counts/limits TCP sockets, which I kept hitting for no apparent reason. On a non virtualized host, or a host that doesn't have that sort of limit the problem isn't really a problem if the FD limits are set fairly high. I was seeing about 5 orphaned sockets per request of a record detail page.

The orphaned sockets show up when 'lsof" is run, as type "sock" with a message of " can't identify protocol"

To confirm this problem, run "lsof | grep " sock " | wc -l" A fixed system will have 2 x <Num Apache Processes> + ~7 of these orphaned sockets. Each apache process has 2 from startup, when a socket call is made without a connect, probably for IPV4/IPV6 setup reasons.

My system would show several hundred to several thousand of the orphaned sockets after a day of running. After restarting apache the number would go back down. And then after loading a record detail page (http://egcatalog/eg/opac/record/10) the number would climb after each request.

The fix is really simple, just call close after the shutdown. Patch on the way.


Tags: pullrequest
Revision history for this message
Josh Stompro (u-launchpad-stompro-org) wrote :
tags: added: pullrequest
Revision history for this message
Galen Charlton (gmc) wrote :

Pushed to master, rel_2_8, and rel_2_7. Thanks, Josh!

Changed in evergreen:
milestone: none → 2.9-beta
status: New → Fix Committed
importance: Undecided → Medium
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.