supports_mmap_offset-related crash with Python 2.6

[Ivan Kozik]

The 'supports_mmap_offset == True' behavior leads to a crash on Ubuntu 8.10 x86_64 + Python 2.6.1 running in VMWare.

For the attached traceback, dulwich/pack.py was modified to print 'size' before it does mmap.mmap(...

Everything seems to work fine if supports_mmap_offset is forced to False.

[James Westby (james-w)]

The full arguments are:

<ivan> PARAMS ARE 6 230952 1 0finding revisions to fetch 0
<ivan> PARAMS ARE 10 2595068 1 12ding revisions to fetch 6
<ivan> bzr: ERROR: mmap.error: [Errno 22] Invalid argument
<ivan> the param order for the print matches mmap.mmap(f.fileno(), size, access=access, offset=offset)

Thanks,

James

[Colin Watson (cjwatson)]

FWIW, this doesn't seem to have fixed my problem. I cobbled together and installed a dulwich package to try this out (lp:~jelmer/dulwich/debian r199, with r226 of http://people.samba.org/bzr/jelmer/dulwich/trunk/ applied on top of this; also lp:~bzr/bzr-git/trunk r340 since that seems to be the most recent version that works with the version of bzr I have). I also made a change similar to Ivan's above to print out the parameters passed to mmap.mmap(). The result was as follows:

$ bzr get git://git.kernel.org/pub/scm/linux/hotplug/udev.git
/usr/lib/python2.6/dist-packages/dulwich/objects.py:25: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
6 4517536 1 0 ] git: Total 24442 (delta 13944), reused 24183 (delta 13760) 0
6 4056275 1 112
bzr: ERROR: mmap.error: [Errno 22] Invalid argument

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 716, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 911, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 547, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1100, in run
    source_branch=br_from)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1134, in sprout
    result_repo.fetch(source_repository, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1133, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/home/cjwatson/.bazaar/plugins/git/fetch.py", line 353, in fetch
    mapping=mapping, fetch_spec=fetch_spec)
  File "/home/cjwatson/.bazaar/plugins/git/fetch.py", line 373, in fetch_refs
    self.fetch_objects(determine_wants, mapping, pb)
  File "/home/cjwatson/.bazaar/plugins/git/fetch.py", line 408, in fetch_objects
    target_git_object_retriever, recorded_wants, pb)
  File "/home/cjwatson/.bazaar/plugins/git/fetch.py", line 278, in import_git_objects
    o = object_iter[head]
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 953, in __getitem__
    type, uncomp = self.get_raw(sha1)
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 943, in get_raw
    offset = self.idx.object_index(sha1)
  File "/home/cjwatson/.bazaar/plugins/git/remote.py", line 192, in idx
    self._data.create_index_v2(self._idx_path, self.resolve_ext_ref)
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 551, in create_index_v2
    entries = self.sorted_entries(resolve_ext_ref)
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 542, in sorted_entries
    ret = list(self.iterentries(resolve_ext_ref))
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 529, in iterentries
    type, obj = self.resolve_object(offset, type, obj, get_ref_text)
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 480, in resolve_object
    type, base_obj = get_offset(base_offset)
  File "/usr/lib/python2.6/dist-packages/dulwich/pack.py", line 572, in get_object_at
    map, map_offset = simp...

[Jelmer Vernooij (jelmer)]

I've removed support for the offset parameter for now, as I can't appropriately test it (I don't have Python 2.6 myself). From what I've seen it hasn't worked appropriately yet anyway, and it would in theory only help with very large pack files.