functional tests leak memory

[Marius Gedminas]

All functional tests in Zope-3-based apps leak memory by keeping references to all in-memory databases created for each test. This was observed with zope.app.testing 3.4.1.

FunctionalTestSetup.setUp() calls multi_database which registers the new database object as a global utility providing IDatabase. The utility registry doesn't automatically remove overriden registrations, it always adds to them.
The object graph keeping the DB objects in memory after running 5 functional tests can be seen at http://mg.pov.lt/db-objects-in-memory.png

The fix is to unregister the IDatabase utilities in FunctionalTestSetup.tearDown(). I'm attaching a tested patch.

[Marius Gedminas (mgedmin)]

Some numbers: on an app I've got here with ~250 functional tests, this patch reduces test runner's memory usage from 500 megs down to 110 megs, and reduces the test run time from 29 minutes down to 21 minutes.

[Marius Gedminas (mgedmin)]

Fix committed in rev 88823.

I'd like to release 3.4.3 with this fix.

[Fred Drake (fdrake)]

+1 on releasing.

[Marius Gedminas (mgedmin)]

Fix released in zope.app.testing 3.4.3.

[Marius Gedminas (mgedmin)]

I'd like to get this into the KGS as well, but apparently I don't have scp access to download.zope.org:/var/www/download.zope.org/zope3.4/controlled-packages.cfg

[Stephan Richter (srichter)]

On Friday 25 July 2008, Marius Gedminas wrote:
> I'd like to get this into the KGS as well, but apparently I don't have
> scp access to download.zope.org:/var/www/download.zope.org/zope3.4
> /controlled-packages.cfg

Darn. I am back in the office on Monday and will try to debug this with you
and Jim.

Regards,
Stephan
--
Stephan Richter
Web Software Design, Development and Training
Google me. "Zope Stephan Richter"