_ConnectionPool._reduce_size provides no hook

[John Eikenberry]

Tracked down a local memory leak to Zope leaking database connections when the _ConnectionPool is temporarily expanded beyond the target size. When the pool is resized the mysql connections are leaked. This is a known limitation with mysqldb, that you must explicitally close() a connection or it will leak. But there is no hook provided to do this.

Unless I'm missing some way of doing this (always possible) I think providing a hook here would really help when dealing with not quite well behaved C modules.

[John Eikenberry (jae)]

After thinking about this some more I don't think a callback/hook would work here as there are no guarantees that the connection is not still in use at this point.

Maybe I could check for this in a synchronizer. Detect that the connection has been removed from the pool and close the database connection. Detecting that the connection has been removed from the pool is the trick though.

[John Eikenberry (jae)]

> = Comment - Entry #3 by tseaver on Jun 13, 2007 9:16 am
>
> One possibility would be to set the "target" size of connections
> which are removed from the pool to 0, and just allow ZODB's
> garbage collection to do its thing for you.

This is what we have done for the moment. Made sure all our sites have the pool-size set to be >= to the zserver-threads value. This fixes our sites but I am helping maintain the ZMySQLDA and would like to include a fix there other than requiring a certain configuration setup.

> We might also add an event to be published at removal (maybe
> at add, as well); you could then register a subscriber which
> used the about-to-be removed ZODB connection to find the DA and
> issued the 'close'.

This sounds like just what I need.
+1

[Tres Seaver (tseaver)]

One possibility would be to set the "target" size of connections
which are removed from the pool to 0, and just allow ZODB's
garbage collection to do its thing for you.

We might also add an event to be published at removal (maybe
at add, as well); you could then register a subscriber which
used the about-to-be removed ZODB connection to find the DA and
issued the 'close'.