SVN branch out of memory

[niels]

I try to checkout a great svn -repository with ca 97000 Revisions. I was able to branch the first 50000 Revisions, but I have no chance to pull the rest. I run it with -Dmem_dump and get

bzr: out of memory
Memory dumped to c:\users\niels\appdata\local\temp\bzr_memdumpvsa5ak.json
Traceback (most recent call last):
  File "bzr", line 143, in <module>
  File "bzrlib\library_state.pyo", line 100, in __exit__
  File "bzrlib\lazy_import.pyo", line 118, in __setattr__
  File "bzrlib\lazy_import.pyo", line 91, in _replace
bzrlib.errors.IllegalUseOfScopeReplacer: ScopeReplacer object 'ui' was used incorrectly: Object tried to replace itself,
 check it's not using its own scope.

The file has only a size of 2.035.420.925 Bytes. Running on a system with 8GB I think there should be more memory available.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/21/2011 8:41 PM, niels wrote:
> Public bug reported:
>
> I try to checkout a great svn -repository with ca 97000 Revisions. I was
> able to branch the first 50000 Revisions, but I have no chance to pull
> the rest. I run it with -Dmem_dump and get
>
> bzr: out of memory
> Memory dumped to c:\users\niels\appdata\local\temp\bzr_memdumpvsa5ak.json
> Traceback (most recent call last):
> File "bzr", line 143, in <module>
> File "bzrlib\library_state.pyo", line 100, in __exit__
> File "bzrlib\lazy_import.pyo", line 118, in __setattr__
> File "bzrlib\lazy_import.pyo", line 91, in _replace
> bzrlib.errors.IllegalUseOfScopeReplacer: ScopeReplacer object 'ui' was used incorrectly: Object tried to replace itself,
> check it's not using its own scope.
>
> The file has only a size of 2.035.420.925 Bytes. Running on a system
> with 8GB I think there should be more memory available.

Mem-dump dumps a description of memory. It doesn't dump the raw bytes of
each object. So it is possible that you are referencing more than just
2GB in the raw content.

I'm guessing this is more bzr-svn, which I think Jelmer has a couple
fixes for.

 affects bzr-svn

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4oeS4ACgkQJdeBCYSNAAN4RwCgrrLJ5iFKz6zp0YePZvatgmBS
i9AAoKFEYikj6zyP480tnpWQrSbUfMv0
=xBCF
-----END PGP SIGNATURE-----

[niels (opensource21)]

The first test was made with bzr 2.3.3. I make the same test with bzr 2.3.4 (svn 1.0.5dev) again (to give you the exact version information. Now I get a less detailed message

bzr: out of memory
Dumping memory requires meliae module.

No idea from where I can get the "meliae module".

[Jelmer Vernooij (jelmer)]

The meliae module is available from http://launchpad.net/meliae

If you are running Debian or Ubuntu you should be able to get it by installing the python-meliae package.

bzr 2.4 and bzr-svn 1.1.0 are significantly better in terms of memory usage.

A workaround might be to pull the revisions in smaller sets, e.g.:

bzr init

bzr pull -r10000 svn://url/to/svn/repo
bzr pull -r20000 svn://url/to/svn/repo
bzr pull -r30000 svn://url/to/svn/repo
....

[niels (opensource21)]

Hi unfortunately I'm on a windows pc, but meliae wouldn't fix the problem so it's not so important to me, but perhaps it should become part of the bazaar distribution.

The workaround doesn't work. I tried
bzr branch -r1000 svn://url/to/svn/repo svn
cd svn
bzr pull -r2000 and it doesn't work

what is quiet curious, because bzr branch -r 50000 svn://url/to/svn/repo svn50000 works.

Unfortunately the windows-installation with no admin rights is gone, so I can't simple test the version 2.4. The only way I know at the moment is to install 2.4. Test it, and then go back to stable version 2.3, which ends in 2 reboots :-(

[niels (opensource21)]

Problem still happens with svn 1.1.0dev from Bazaar (bzr) 2.4b5.