bzr commit will break when given too many files as arguments

[James Troup]

bzr commit will break when given too many (where too many means more than allowed by ulimit -n) files as an argument, e.g.

$ rm -fr test; mkdir test; cd test; bzr init
$ for i in $(seq 1 1200); do echo a > ${i}; done; bzr add; bzr ci -m "init"
$ for i in $(seq 1 1200); do echo b > ${i}; done; bzr ci -m "this will work"
$ for i in $(seq 1 1200); do echo c > ${i}; done; bzr ci -m "this won't" *

The last command will error with something like this:

aborting commit write group: IOError(24, 'Too many open files')
bzr: ERROR: [Errno 24] Too many open files: u'/home/james/scratch/test/118'

[James Troup (elmo)]

Oh, sorry, this is using bzr 2.1.1-1 on Ubuntu 10.04 with the default max open FD limit of 1024.

[Parth Malwankar (parthm)]

I see the same behavior with trunk on lucid.

aborting commit write group: IOError(24, 'Too many open files')
bzr: ERROR: [Errno 24] Too many open files: u'/home/parthm/tmp/x/118'
[x]%

[Parth Malwankar (parthm)]

aborting commit write group: IOError(24, 'Too many open files')
bzr: ERROR: exceptions.IOError: [Errno 24] Too many open files: u'/home/parthm/tmp/x/118'

Traceback (most recent call last):
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commands.py", line 909, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commands.py", line 1107, in run_bzr
    ret = run(*run_argv)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commands.py", line 687, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commands.py", line 702, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/builtins.py", line 3184, in run
    exclude=safe_relpath_files(tree, exclude))
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/workingtree_4.py", line 197, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/mutabletree.py", line 200, in commit
    *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commit.py", line 286, in commit
    possible_master_transports=possible_master_transports)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/cleanup.py", line 118, in run
    self.cleanups, self.func, self, *args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commit.py", line 415, in _commit
    self._update_builder_with_changes()
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commit.py", line 650, in _update_builder_with_changes
    self.work_tree, self.basis_revid, iter_changes):
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/repository.py", line 655, in record_iter_changes
    for change in iter_changes:
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/commit.py", line 671, in _filter_iter_changes
    for change in iter_changes:
  File "_dirstate_helpers_pyx.pyx", line 1410, in bzrlib._dirstate_helpers_pyx.ProcessEntryC.__next__
  File "_dirstate_helpers_pyx.pyx", line 1689, in bzrlib._dirstate_helpers_pyx.ProcessEntryC._iter_next
  File "_dirstate_helpers_pyx.pyx", line 1522, in bzrlib._dirstate_helpers_pyx.ProcessEntryC._iter_next
  File "_dirstate_helpers_pyx.pyx", line 1181, in bzrlib._dirstate_helpers_pyx.ProcessEntryC._process_entry
  File "/storage/parth/src/bzr.dev/trunk/bzrlib/workingtree_4.py", line 1311, in stat_and_sha1
    file_obj = file(abspath, 'rb', 65000)
IOError: [Errno 24] Too many open files: u'/home/parthm/tmp/x/118'
[x]%