Robert Collins wrote:
> On Mon, 2008-08-18 at 18:35 +0000, John A Meinel wrote:
>> The attached patch seems to correct a logic bug. Specifically the
>> variable is "fetch_uses_deltas" but the variable passed is
>> "include_delta_closure". Which actually means "and transmit all history
>> and use full-texts". Which "fetch_uses_deltas" seems to have the
>> opposite meaning.
>>
>> With the 'not' fix, memory consumption drops to 100MB. (Instead of 700+MB).
>> However, this only applies to bzr.dev, so it shouldn't actually be a regression in 1.6.
>> Also, the target repository is still grossly bloated (140MB).
>>
>> If I change the fetch order for pack_repo to also be "topological" then
>> I get the expected target size of 35MB. I'm not as convinced that is the
>> right fix, but it does make the fetch correct.
>
> VF shouldn't be converting texts to full-text willy-nilly. It can't with
> the uses_deltas set appropriately anyway; it has to have the basis text
> locally to be able to convert via a fulltext.
>
> -Rob
So the specific issue was that I wasn't setting the _use_deltas for all
objects. I was setting it for .texts, but I missed the .inventories. And, as
it turns out with this repo, the .inventories was bloating more than the rest.
(This repo is about 6k small files, so a large inventory causes more bloat
than extra file fulltexts.)
Which is why my final fix set it to 'unordered'. (Rather than 'unsorted',
because get_record_stream() isn't supposed to know what 'unsorted' means. :)
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Robert Collins wrote: delta_closure" . Which actually means "and transmit all history
> On Mon, 2008-08-18 at 18:35 +0000, John A Meinel wrote:
>> The attached patch seems to correct a logic bug. Specifically the
>> variable is "fetch_uses_deltas" but the variable passed is
>> "include_
>> and use full-texts". Which "fetch_uses_deltas" seems to have the
>> opposite meaning.
>>
>> With the 'not' fix, memory consumption drops to 100MB. (Instead of 700+MB).
>> However, this only applies to bzr.dev, so it shouldn't actually be a regression in 1.6.
>> Also, the target repository is still grossly bloated (140MB).
>>
>> If I change the fetch order for pack_repo to also be "topological" then
>> I get the expected target size of 35MB. I'm not as convinced that is the
>> right fix, but it does make the fetch correct.
>
> VF shouldn't be converting texts to full-text willy-nilly. It can't with
> the uses_deltas set appropriately anyway; it has to have the basis text
> locally to be able to convert via a fulltext.
>
> -Rob
So the specific issue was that I wasn't setting the _use_deltas for all
objects. I was setting it for .texts, but I missed the .inventories. And, as
it turns out with this repo, the .inventories was bloating more than the rest.
(This repo is about 6k small files, so a large inventory causes more bloat
than extra file fulltexts.)
Which is why my final fix set it to 'unordered'. (Rather than 'unsorted',
because get_record_stream() isn't supposed to know what 'unsorted' means. :)
John enigmail. mozdev. org
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://
iD8DBQFIqfpLJde BCYSNAAMRAmiiAJ 9qKJ/IwCuWWf5W7 i3q5Br/ R8s/dgCdFYXQ ++AGTjdg=
fsvtq+YJ15gqFDw
=Yml+
-----END PGP SIGNATURE-----