File opened by mmap is not closed causes removing file failure

[nothize]

This is like an issue for Windows because from my personal experience, file deletion on Linux won't be affected by opened file lock. But on Windows opened files cannot be deleted.

Related id: 557585, 565448.

I've downloaded Dulwich 0.6.0 and try out with the hg-git extension.

Combing Python debug mode with Anatoly Techtonik's "monkeypatching" code and file handle dumper, here is the result:

Output of printOpenFiles():

### 3 OPEN FILES: [R:\hg\inspection\.hg\hgrc, R:\hg\inspection\.hg\git\objects\pack\pack-1a59751d438961fde15357f30146e30cc612fe34.pack, R:\hg\inspection\.hg\git\objects\pack\tmpqxvqyi.pack]

Partial output of handle -p hg.exe:

   dc: File R:\hg\inspection
  1e0: File R:\hg\inspection\.hg\git\objects\pack\pack-1a59751d438961fde15357f30146e30cc612fe34.idx
  1e8: File R:\hg\inspection\.hg\git\objects\pack\pack-fcdb717aa8f634411314e2b8288f17333da2ed44.idx
  1ec: File R:\hg\inspection\.hg\git\objects\pack\pack-1a59751d438961fde15357f30146e30cc612fe34.pack
  1f0: File R:\hg\inspection\.hg\git\objects\pack\tmprkzdib.pack
  384: File R:\hg\inspection\.hg\hgrc

It shows that the files ending in .idx are so special that doesn't appear in the Python opened file list.

By debugging deep, it is found that the use of mmap.mmap(...) in pack.py(167)_load_file_contents would keep a file handle and there is no place to close the mmap. Perhaps this is where the file handle leak happen.

[nothize (nothize)]

It is inspected that the dulwich/pack.py file have two path way, one uses mmap and another not.

By disabling the use of mmap() has resolved the problem since mmap() opens the file and dulwich has no where to close it.

A quick look to the code shows that it's rather quite complicated to determine a proper point for closing the file opened by mmap. Therefore, a temporary resolution of disabling mmap maybe satisfactory when the running platform is Windows.

[rctay (rctay89)]

A couple of close()'s have been recently committed, which resolves this issue.

[Daniel Dabrowski (rodpl)]

I've still got the same problem.
idx file is not closed.
I'm acctually using this code version (2011-01-05)

https://github.com/jelmer/dulwich/commit/f841bf00fd930132c9a41603b4dd7ff2e64dbe00

Problem exists on Windows 7 x64 witn Python 2.7 x64 mercurial with hg-git extension

I've attached log from debugger

[Balsaad (balsaad)]

I am using TortoiseHG 1.1.9.1 (Hg 1.7.5) and hg-git r372 from https://bitbucket.org/durin42/hg-git on Windows XP SP3.

I updated dulwich to 0.7.0 and I still get the "abort: The process cannot access the file because it is being used by another process" message. If I make similar changes as shown in comment #2, hg pull succeeds (at least no abort message and the info looks okay - I haven't used extensively.)