MemoryError on diff/commit due to corrupted dirstate

Bug #186014 reported by laga on 2008-01-25
22
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Bazaar
High
John A Meinel

Bug Description

Hi,

whenever I run bzr diff/commit/status on my local branch, bzr 1.1-1~bazaar1~gutsy1 (from PPA) will just error out. This also happened with 1.0 which I got from the repository listed on bazaar-cvs.org.

The checkout with which I'm experiencing problems used to work fine and I don't remember having changed anything (except for a few files in the checkout, of course). The checkout holds less than 1500 files (not sur since some files in that directory haven't been added yet). Its size is about 8Mb.

Other checkouts still work fine.

Here's the log:

bzr diff
bzr: ERROR: exceptions.MemoryError:

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 806, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 762, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 772, in ignore_pipe
    result = func(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 1512, in run
    extra_trees=extra_trees, using=using)
  File "/usr/lib/python2.5/site-packages/bzrlib/diff.py", line 505, in show_diff_trees
    return differ.show_diff(specific_files, extra_trees)
  File "/usr/lib/python2.5/site-packages/bzrlib/diff.py", line 925, in show_diff
    return self._show_diff(specific_files, extra_trees)
  File "/usr/lib/python2.5/site-packages/bzrlib/diff.py", line 948, in _show_diff
    executable) in sorted(iterator, key=changes_key):
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1793, in _iter_changes
    state._read_dirblocks_if_needed()
  File "/usr/lib/python2.5/site-packages/bzrlib/dirstate.py", line 1877, in _read_dirblocks_if_needed
    _read_dirblocks(self)
  File "_dirstate_helpers_c.pyx", line 694, in _dirstate_helpers_c._read_dirblocks_c
  File "_dirstate_helpers_c.pyx", line 661, in _dirstate_helpers_c.Reader._parse_dirblocks
  File "_dirstate_helpers_c.pyx", line 626, in _dirstate_helpers_c.Reader._get_entry
MemoryError

bzr 1.1.0 on python 2.5.1.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'diff']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'de_DE.UTF-8'
plugins:
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.5/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

Regards,

Michael

Related branches

James Westby (james-w) wrote :

Hi,

Thanks for your bug report.

I am interested in what is causing the problem that you are
seeing. From the traceback I would guess that it is using a
lot of memory to read the dirstate (the on disk representation
of the state of the working tree).

Is this an open source project? If so can we have a copy of
the code to try and reproduce this. If not could we just have
the dirstate? It lists the filenames, but should not give any
more information than that. Failing that just the size
of the dirstate file would a start (ls -l .bzr/checkout/dirstate)

Is there any improvement if you branch the project to a different
location (on the same machine) and then make the changes?

Thanks,

James

On Mon, 2008-01-28 at 22:28 +0000, James Westby wrote:
> Hi,
>
> Thanks for your bug report.
>
> I am interested in what is causing the problem that you are
> seeing. From the traceback I would guess that it is using a
> lot of memory to read the dirstate (the on disk representation
> of the state of the working tree).

Actually, I suspect its a corrupt dirstate; using the non C version of
the dirstate parser will likely work better and give a better message.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

James Westby (james-w) wrote :

On Tue, 2008-01-29 at 11:35 +0000, Robert Collins wrote:
> On Mon, 2008-01-28 at 22:28 +0000, James Westby wrote:
> > Hi,
> >
> > Thanks for your bug report.
> >
> > I am interested in what is causing the problem that you are
> > seeing. From the traceback I would guess that it is using a
> > lot of memory to read the dirstate (the on disk representation
> > of the state of the working tree).
>
> Actually, I suspect its a corrupt dirstate; using the non C version of
> the dirstate parser will likely work better and give a better message.

Michael, in order to test this, could you please either delete or rename
/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_c*

This will mean that the python versions are used instead, hopefully
giving us some more information.

Everyone else, Is it worth adding an environment variable to override
the C versions and use the python versions for testing this sort of
thing in future?

Thanks,

James

I just commented on #153749 about this but I get the same error. It started for me when my computer locked up in the middle of a "bzr status" command. When I rebooted I started getting this error for most bzr commands.

Attached is my .bzr/checkout/dirstate

Here is the message I get using the non C version of the dirstate helper:

------------------
bzr: ERROR: exceptions.AssertionError:

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 806, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 762, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 772, in ignore_pipe
    result = func(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 189, in run
    to_file=self.outf, short=short, versioned=versioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/status.py", line 114, in show_tree_status
    want_unversioned=want_unversioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/tree.py", line 93, in changes_from
    want_unversioned=want_unversioned,
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 127, in read_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/tree.py", line 707, in compare
    want_unversioned=want_unversioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/delta.py", line 217, in _compare_trees
    want_unversioned=want_unversioned):
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1793, in _iter_changes
    state._read_dirblocks_if_needed()
  File "/usr/lib/python2.5/site-packages/bzrlib/dirstate.py", line 1877, in _read_dirblocks_if_needed
    _read_dirblocks(self)
  File "/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_py.py", line 203, in _read_dirblocks_py
    assert trailing == ''
AssertionError

bzr 1.1.0 on python 2.5.1.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'status']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.1.0]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.93.0]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.5/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.
-----------------------------------------

brettatoms (brettatoms) wrote :

When trying to branch this same directory I get the following error which is the same as my previous comment except for the last line about the lock:

bzr: ERROR: exceptions.AssertionError:

(...same as previous comment...)

  File "/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_py.py", line 203, in _read_dirblocks_py
    assert trailing == ''
AssertionError

bzr 1.1.0 on python 2.5.1.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'branch', 'trunk', 'trunk2']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.1.0]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.93.0]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.5/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.
/usr/lib/python2.5/site-packages/bzrlib/lock.py:79: UserWarning: lock on <open file u'/home/brett/devel/bauble/trunk/.bzr/checkout/dirstate', mode 'rb' at 0x8786188> not released
  warn("lock on %r not released" % self.f)

brettatoms (brettatoms) wrote :

Is there a quick fix for this? At the moment I can't commit to my tree and so my work is stalled.

On Wed, 2008-02-06 at 17:03 +0000, brettatoms wrote:
> Is there a quick fix for this? At the moment I can't commit to my tree
> and so my work is stalled.
>

My best suggestion is to branch to somewhere else on your system and
then copy over the files that have changed.

I am going to look in the problem tonight, so I may be able to give
you a fix for your dirstate file.

Thanks,

James

It sounds like your dirstate file itself is corrupted, so you need to generate a new working tree and start there. There are a couple of things you could try, but step one is definitely

Backup the current work (a simple cp -a, or tarball would be sufficient)

The way I would do it is to create a new branch with:

cd ..
bzr branch project new_project

then copy over your changes

rsync --exclude .bzr project/ new_project

You may need to redo some stuff like renames, add, etc. But it should let you get back to work.

On Wed, 2008-02-06 at 17:03 +0000, brettatoms wrote:
> Is there a quick fix for this? At the moment I can't commit to my tree
> and so my work is stalled.
>

Sorry, just seen your other message saying that you can't branch
from this branch, so my suggestion will not work.

If you have an ssh server on that machine then branching over ssh
may work. Alternatively bzr serve and using bzr:// should accomplish the
same.

Thanks,

James

Hi,

The last line of your dirstate reads

^@web/images/screenshots/0.6^@bauble-sqlite-conn-dialog-0.6.0.png^@webimagesscreenshots-20070625115140-f77u2cc78wp550ne-730^@f^@df91fbb396872aea3bfc1ad609e73f277260d755^@28051^@y^@AABtk0egWKxHoFitAAADBgYIr7IAAIHt^@f^@df91fbb396872aea3bfc1ad609e73f277260d755^@28051^@y^@brettatoms@g

which is truncated.

Is there any chance that you had disk space issues, or that you suffered a power failure
or disk errors around the time that this problem occured?

Thanks,

James

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John A Meinel wrote:
> It sounds like your dirstate file itself is corrupted, so you need to
> generate a new working tree and start there. There are a couple of
> things you could try, but step one is definitely
>
> Backup the current work (a simple cp -a, or tarball would be sufficient)
>
> The way I would do it is to create a new branch with:
>
> cd ..
> bzr branch project new_project

Unfortunately, branch now opens working trees for the file copy
acceleration. So branch will break, too. I think push would work.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHqffy0F+nu1YWqI0RAgWwAJ998tVvy7mF3J2g2dr8eVo48aGmNgCfZN4k
Wq7tqE6D6SXckArz5TukJSE=
=hsD8
-----END PGP SIGNATURE-----

Matthew Fuller (fullermd) wrote :

On Wed, Feb 06, 2008 at 06:09:55PM -0000 I heard the voice of
Aaron Bentley, and lo! it spake thus:
>
> Unfortunately, branch now opens working trees for the file copy
> acceleration. So branch will break, too. I think push would work.

You could probably also rm (or mv) .bzr/checkout/ and then branch (and
maybe copy back the .bzr/checkout/ from the new branch).

It would be nice to have a "bzr blat-out-pristine-dirstate" command
to streamline recovery from this sort of thing...

--
Matthew Fuller (MF4839) | <email address hidden>
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
           On the Internet, nobody can hear you scream.

here from bug #195231

John A Meinel (jameinel) wrote :

I have a patch pending for at least giving better error reporting rather than getting MemoryError or a segfault.

Changed in bzr:
assignee: nobody → jameinel
importance: Undecided → High
milestone: none → 1.7
status: New → Fix Committed
Christian Reis (kiko) wrote :

Hmmm! Disk space issues! I hadn't thought of that!

Jelmer Vernooij (jelmer) on 2009-01-28
Changed in bzr:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers