Comment 5 for bug 494406

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John A Meinel wrote:
> John A Meinel wrote:
>> Gareth White wrote:
>>> The memory usage *linearly* increases with the number of commits. I just
>>> did a test involving 999 "unchanged" commits to an empty shared
>>> repository and the server memory usage reached 500,008 KB. Since it
>>> started at 14,564 this represents an average of 485 KB increase per
>>> commit.
>>> That was with 2.0.1 on the server and client. With 2.1.0b3 it went from
>>> 13,996 to 279,840 KB after 999 commits (280 KB per commit). This was on
>>> a separate empty shared repository.
>>> Are you able to reproduce this behaviour?
>
>> So it wasn't something that was linear at all. I did run a loop with
>> ~700 --unchanged commits. I watched it a bit at the beginning and it was
>> pretty much fixed memory consumption at about 84MB. When I came back
>> later, it had jumped to 237MB (295MB peak).
>
>> I haven't found any specific cause for this, though I suspect something
>> with the autopacking code. (Given that it was steady most of the time,
>> and then jumped at some point.)
>
>> John
>> =:->
>
> I also wanted to comment that the memory does indeed seem to have
> 'leaked'. At least, a memory dump from meliae only shows about 12MB of
> referenced memory, which is a far cry from the 250MB reported by the OS.
>
> John
> =:->

Further info.

I can reproduce this with a 'bzr://' server running bzr.dev, and a
simple python loop around "wt.commit()". I'm pretty darn sure that I'm
seeing the memory increase only when the server tries to autopack.

If everything is done in the local process, I *don't* see the memory go up.

Specifically, if I hold the lock for all commits, looping over
'wt.commit()' starting at 33.6MB (42MB peak), at the end of 1000
commits, I was at 50.7kB peak and 42.5kB resident.

Doing the same thing on the server seemed to result in about 240MB consumed.

Note that I also encountered bug #495588 (commit() in a loop slows down
if you hold a write lock because get_parent_map() suddenly transmits
large amounts of data on each commit.) However, this can be demonstrated
with the command line client, which would be opening a new connection
for each commit. So whatever is leaking memory is triggered by the rpc,
not by a cache being held and then 'lost'.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksimQUACgkQJdeBCYSNAAN2ZwCgw1Kgc3nRtqGuZSyUdyF7T0nF
sIwAnA4p+B7/rO8lxeRe+GSmZCvY44BG
=jfpo
-----END PGP SIGNATURE-----