Loggerhead is too slow and consumes gigabytes of memory

[Nicholas Allen]

I am using the latest version of loggerhead to view a branch on our server. The branch only has about 50 revisions but has a large number of files (it contains the boost source code).

Loggerhead consumes over 3GB of RAM and takes 20 minutes without displaying a single revision for this branch. Finally it crashes after all memory is exhausted.

[Michael Hudson-Doyle (mwhudson)]

We know, at least in general. This sounds exceptionally bad though.

Some questions:

 - which version of bazaar are you using?
 - what format is the branch in?
 - is the branch publicly available?

Can you attach the traceback of the failure? That might let me figure out which part of the process is using so much RAM.

[Nicholas Allen (nick-allen)]

Bazaar 1.12.
Repository was in Pack 0.92.
Unfortunately, the branch is not open source code so I can't give a link to it. I will try to post a traceback.

After upgrading it to 1.9 it still consumed large amounts of RAM and took about 5-6 minutes but then it finally displayed the log! But this t is still too long as it does this every time and not just the first time. My server is almost brought to its knees trying to display the log.

I am happy to help profile it if I can offer any other information...

[Michael Hudson-Doyle (mwhudson)]

Ah, is one of the revisions that loggerhead is trying to display a large initial import? That would make this a duplicate of https://bugs.edge.launchpad.net/loggerhead/+bug/253950.

We should only display a truncated list if the # of added files is greater than some threshold.

[Nicholas Allen (nick-allen)]

Yes one of the revisions imported the boost C++ library which is a huge number of files. Any idea when this issue will be fixed?

[Michael Hudson-Doyle (mwhudson)]

Hopefully tomorrow :)