Comment 3 for bug 723234

Yes i put a couple of huge files, like 200MB,
and media file size is like 300KB

the commit do fine, and i dont have problem sinse i try to make push with
sftp, then i thy with file push
and after with check, all fails.

The I init new repositorie without huge files, pulls looks good, but i wait
like an our and it not finish

On Tue, Feb 22, 2011 at 9:34 PM, Andrew Bennetts <
<email address hidden>> wrote:

> My guess is the root cause of this is an out-of-memory problem (so is
> probably a duplicate of one of the existing reports tagged 'memory')..
> Does your big tree include any particularly big files?
>
> That whatever happens causes SystemError from the zlib module would be a
> bug in Python itself. Calling zlib.decompress should never cause that.
>
> Objects/stringobject.c:4271 of Python 2.6.6 is in _PyString_Resize,
> after this check is tripped:
>
> (!PyString_Check(v) || Py_REFCNT(v) != 1 || newsize < 0 ||
> PyString_CHECK_INTERNED(v))
>
> zlibmodule.c does call _PyString_Resize directly in places, including in
> the decompress function (PyZlib_decompress)... at a glance I'd suspect
> the newsize < 0 is the issue: if the string being decompressed expands
> to something sufficiently huge I think that could happen.
> PyZlib_decompress calls _PyString_Resize to double the size of the
> buffer (via << 1) it is decompressing into as the zlib library produces
> more decompressed data, and newsize is a Py_ssize_t, a signed type...
>
> If that's the case, then basically we need to either fix bzr to either
> extract such large compressed strings in parts rather than all at once,
> or avoid compressing such large strings in the first place. There's
> been some discussion about and gradual progress towards these solutions.
> John would know more about how far off we are.
>
> Out of interest, how big is your "big tree"? How many files, and how
> big is the biggest file, and what's the average file size?
>
> ** Changed in: bzr
> Importance: Undecided => Medium
>
> ** Changed in: bzr
> Status: New => Confirmed
>
> ** Summary changed:
>
> - cant pull or check, with big tree
> + Big tree causes 'SystemError: ../Objects/stringobject.c' in groupcompress
>
> ** Tags added: memory
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/723234
>
> Title:
> Big tree causes 'SystemError: ../Objects/stringobject.c' in
> groupcompress
>
> Status in Bazaar Version Control System:
> Confirmed
>
> Bug description:
> i add a big tree and before commit, y cant pull, or check,
>
> bzr check
> Checking working tree at '/home/sandrade/workspace'.
> Checking branch at 'file:///home/sandrade/workspace/'.
> Checking repository at 'file:///home/sandrade/workspace/'.
> bzr: failed to report crash using apport:
> OSError(13, 'Permiso denegado')
> bzr: ERROR: exceptions.SystemError: ../Objects/stringobject.c:4271: bad
> argument to internal function
>
> Traceback (most recent call last):
> File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 912, in
> exception_to_return_code
> return the_callable(*args, **kwargs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1112,
> in run_bzr
> ret = run(*run_argv)
> File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 690, in
> run_argv_aliases
> return self.run(**all_cmd_args)
> File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 705, in
> run
> return self._operation.run_simple(*args, **kwargs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 135, in
> run_simple
> self.cleanups, self.func, *args, **kwargs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 165, in
> _do_with_cleanups
> result = func(*args, **kwargs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 3286,
> in run
> check_dwim(path, verbose, do_branch=branch, do_repo=repo,
> do_tree=tree)
> File "/usr/lib/python2.6/dist-packages/bzrlib/check.py", line 452, in
> check_dwim
> check_repo=do_repo)
> File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 140,
> in read_locked
> result = unbound(self, *args, **kwargs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 2776,
> in check
> check_repo=check_repo)
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 2780,
> in _check
> result.check(callback_refs)
> File "/usr/lib/python2.6/dist-packages/bzrlib/check.py", line 98, in
> check
> self.repository._check_inventories(self)
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1206,
> in _check_inventories
> self._do_check_inventories(checker, bar)
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1263,
> in _do_check_inventories
> checker, last_object, current_keys[(kind,) + record.key])
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1289,
> in _check_record
> self._check_text(record, checker, item_data)
> File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1303,
> in _check_text
> content = record.get_bytes_as('fulltext')
> File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line
> 438, in get_bytes_as
> self._manager._prepare_for_extract()
> File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line
> 538, in _prepare_for_extract
> self._block._ensure_content(self._last_byte)
> File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line
> 151, in _ensure_content
> self._content = zlib.decompress(self._z_content)
> SystemError: ../Objects/stringobject.c:4271: bad argument to internal
> function
>
> bzr 2.2.1 on python 2.6.6
> (Linux-2.6.35-25-generic-pae-i686-with-Ubuntu-10.10-maverick)
> arguments: ['/usr/bin/bzr', 'check']
> encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'es_EC.UTF-8'
> plugins:
> bash_completion
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/bash_completion [2.2.1]
> bzrtools
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/bzrtools [2.2.0]
> explorer
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/explorer [1.1.0]
> launchpad
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/launchpad [2.2.1]
> netrc_credential_store
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/netrc_credential_store
> [2.2.1]
> news_merge
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/news_merge [2.2.1]
> qbzr
> /usr/lib/python2.6/dist-packages/bzrlib/plugins/qbzr [0.19.1]
>
> *** Bazaar has encountered an internal error. This probably indicates a
> bug in Bazaar. You can help us fix it by filing a bug report at
> https://bugs.launchpad.net/bzr/+filebug
> including this traceback and a description of the problem.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/bzr/+bug/723234/+subscribe
>