Comment 3 for bug 303538

We have fixed the code that does smart fetches to not fetch more than 5MB at a time, but IIRC we don't have any code that limits the local fetching.

So it seems we are trying to copy >2GB of data in one read + write, and that breaks fp.read().

As a side note, it appears that you are not using a shared repository, which means we have to copy the 20GB of history around. If you had done "bzr init-repo" at the beginning, we would only need to create new working-trees or very lightweight branches, etc.

As a quick workaround, you could do:

  bzr init-repo
  rm .bzr/repository -rf
  mv existing_branch/.bzr/repository .bzr
  touch .bzr/repository/shared-storage

Note that you only want to do this if you truly *don't* have a shared repository already, otherwise it deletes the existing shared repo.

Anyway, the simple fix is to pass "max_size=XXXX" to Transport._coalesce_offsets. At the moment we use "0" as the default value, which means unlimited. We could make 0 == 2GB, or more reasonably have a harder max-size of 10MB or so. 10MB should be a reasonable buffer size that doesn't hit all of available RAM, and also avoids enough round trips.

As an example, see the attached patch.