branching transfers excessive amounts of data

[Soren Hansen]

I'm currently branching lp:linux.

According to jam, the repo is ~700MB.

The branch command is still running. This is what it currently shows:
1742362kB 361kB/s - Fetching revisions:Inserting stream:Estimate 2203582/2512932

And du linux/.bzr says:
442384 linux/.bzr/

So it (says that it) has transferred 1.7GB to construct a 442MB repo. That seems excessive. Either the transfer-o-meter is lying or something should be done to reduce the amount of data that needs to travel over the network.

[Soren Hansen (soren)]

I'm using bzr+ssh, by the way.

[John A Meinel (jameinel)]

I do wonder if this is a dupe. I'm guessing the bytes transferred is actually accurate. My initial guess is that it is splitting the groups up on the server, streaming them as expanded fulltexts, and getting recombined into efficient delta-compressed content locally.
It doesn't make a lot of sense why it would be 1.7GB for a repository that only has 700MB on disk (at least by browsing to the packs/ page and summing up the sizes I see there.)

[Soren Hansen (soren)]

I did half a bajillion bzr calls while this ran and I weeded their log entries out. I don't think I removed anything that pertains to this.

[Soren Hansen (soren)]

Oh, and the .bzr directory ended up at 609904kB.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/12/2011 12:41 PM, Soren Hansen wrote:
> I did half a bajillion bzr calls while this ran and I weeded their log
> entries out. I don't think I removed anything that pertains to this.
>
> ** Attachment added: "linux-branch.log"
> https://bugs.launchpad.net/bzr/+bug/758620/+attachment/2023488/+files/linux-branch.log
>

It has the part I cared about:
5934.696 Transferred: 2244050kB (378.2kB/s r:2244049kB w:1kB)

Which is that we read 2.2GB of content to repack it into 610MB of actual
content. So a read-amplification of almost 4x.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2kNP0ACgkQJdeBCYSNAANoAQCeIp9MVTTiFMKC6EfC3Jl4iZq0
fBAAn0xtLK3TwDxdoLsNAscaD3za5ycT
=cIRK
-----END PGP SIGNATURE-----