bzr-hg consumes very large amounts of memory during initial download

Bug #610153 reported by Peter Bex on 2010-07-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar Hg Plugin
Medium
Unassigned

Bug Description

While downloading the initial set of revisions, hg-bzr blows up in memory causing Launchpad's bzr integration to fail on importing these two branches:

https://code.launchpad.net/~peter-bex/chamilo/chamilo-hg-mirror

and

https://code.launchpad.net/~peter-bex/chamilo/chamilo-classic-hg-mirror

I got 1.6G of memory used during the process of fetching manifests early on: at 500/4219

See also https://answers.edge.launchpad.net/launchpad-code/+question/118906

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter Bex wrote:
> Public bug reported:
>
> While downloading the initial set of revisions, hg-bzr blows up in
> memory causing Launchpad's bzr integration to fail on importing these
> two branches:
>
> https://code.launchpad.net/~peter-bex/chamilo/chamilo-hg-mirror
>
> and
>
> https://code.launchpad.net/~peter-bex/chamilo/chamilo-classic-hg-mirror
>
> I got 1.6G of memory used during the process of fetching manifests early
> on: at 500/4219
>
> See also https://answers.edge.launchpad.net/launchpad-
> code/+question/118906
>
> ** Affects: bzr-hg
> Importance: Undecided
> Status: New
>

Would you be able to install Meliae (lp:meliae), and then when memory
consumption gets big use either SIGQUIT (Ctrl+\) or SIGBREAK
(Ctrl+Pause/Break) on Windows to get into a debugger. And then run:

 from meliae import scanner
 scanner.dump_all_objects('memory.dump')

And then bzip2 the result and attach it to this bug?

That would make it easier to determine where the memory is actually
being consumed.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxNx+MACgkQJdeBCYSNAAMHiACgmALF4yx8dI3q+o2g/NwHSO3B
CmQAn2dGrIYr+QGwoixf+4QsbXQhzUWl
=umdh
-----END PGP SIGNATURE-----

Jelmer Vernooij (jelmer) wrote :

  status triaged
  importance medium

This is a known issue - because of the way the mercurial smart server
protocol works (it sends revisions first, manifests second and then
contents) the memory usage of is linear with the number of revisions agt
the moment.

This particular repository is pretty large.

Changed in bzr-hg:
importance: Undecided → Medium
status: New → Triaged
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers