This is strictly an issue with the bencode.Encoder class. I haven't
found the bug, as I see it using realloc and free where I would expect
to need it. But doing:
bigstr = 'this is a fairly big string' * 10000
e = bencode_pyx.Encoder()
for i in xrange(10):
e.process(bigstr)
del e
Doing this repeatedly, I see memory consumption grow by about 2.7MB.
Which is exactly the "e.size" value...
I tracked it down. For whatever reason I didn't notice, but the Encoder
class was using __del__ as its deconstructor, but it is supposed to use
__dealloc__
=== modified file 'bzrlib/_bencode_pyx.pyx'
- --- bzrlib/_bencode_pyx.pyx 2009-06-05 01:48:32 +0000
+++ bzrlib/_bencode_pyx.pyx 2009-12-18 21:26:50 +0000
@@ -261,7 +261,7 @@ self.maxsize = maxsize
self.tail = p
dealloc is the C level deallocation, and it always runs, and doesn't
mess with the Python garbage collector.
For whatever reason __del__ is never getting called on this class.
Possibly because it is an extension type. I'll also note that none of
the members of Encoder are 'object' types, which means it doesn't have
to participate in GC the way other classes do.
Anyway, the above patch fixes the looping over encoders bug,
and it appears to fix the memory leak.
status fixcommitted
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
This is strictly an issue with the bencode.Encoder class. I haven't
found the bug, as I see it using realloc and free where I would expect
to need it. But doing:
bigstr = 'this is a fairly big string' * 10000 pyx.Encoder( )
e = bencode_
for i in xrange(10):
e.process(bigstr)
del e
Doing this repeatedly, I see memory consumption grow by about 2.7MB.
Which is exactly the "e.size" value...
I tracked it down. For whatever reason I didn't notice, but the Encoder _bencode_ pyx.pyx' _bencode_ pyx.pyx 2009-06-05 01:48:32 +0000 _bencode_ pyx.pyx 2009-12-18 21:26:50 +0000
self. maxsize = maxsize
class was using __del__ as its deconstructor, but it is supposed to use
__dealloc__
=== modified file 'bzrlib/
- --- bzrlib/
+++ bzrlib/
@@ -261,7 +261,7 @@
self.tail = p
- - def __del__(self):
free( self.buffer)
self. buffer = NULL
self. maxsize = 0
+ def __dealloc__(self):
dealloc is the C level deallocation, and it always runs, and doesn't
mess with the Python garbage collector.
For whatever reason __del__ is never getting called on this class.
Possibly because it is an extension type. I'll also note that none of
the members of Encoder are 'object' types, which means it doesn't have
to participate in GC the way other classes do.
Anyway, the above patch fixes the looping over encoders bug,
and it appears to fix the memory leak.
status fixcommitted
John enigmail. mozdev. org/
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAks r+uQACgkQJdeBCY SNAANpAACfa43sE iAst4cT3h31dkD8 lQy3 z1P8CpWakYNyXyO Hj
i3EAn1fQqBV28TD
=rxMZ
-----END PGP SIGNATURE-----