InconsistentDelta error when using bzr update

[Mark Grandi]

I was trying to commit a new revision in an older branch when bazaar explorer told me the branch was out of date. So i updated it, and then I started getting this error:

InconsistentDelta: An inconsistent delta was supplied involving 'prog1/test cases/252test.pl', '252test.pl-20110216071045-lgametc6cykjk3kr-1'
reason: Unable to find block for this record. Was the parent added?

I thought it was a problem with conflicts so i tried to revert the branch, but then it started giving tons of conflicts and whatnot. I was told to report this as a bug!

I have the branch and the repository (on the server) saved in case anyone needs to look at them, and attached is the bzr.log file that starts with my attempted commit and has all of my failed updates and whatnot

[Mark Grandi (markgrandi)]

Also to note that the repository on the server is not corrupted, checking out a fresh branch from the server repo works fine.

[John A Meinel (jameinel)]

I'm not sure what the cause is, but I figured I'd copy my IRC comments, since I don't think you saw them.

(11:10:56 AM) jam: AuroraBorealis: well, the change seems to indicate that 252test.pl is being added but the directory prog1/testcases doesn't exist.
(11:12:01 AM) jam: You might try "bzr repair-workingtree" in case it is a local issue. Though it would be nice to save a snapshot of .bzr/checkout/dirstate
(11:12:07 AM) jam: in case we can figure out what the difference is.

[Mark Grandi (markgrandi)]

I have a snapshot of both the server repository and the local working tree. Its definitely a local issue since when i did a fresh branch from the server, everything worked fine. Should I upload that here?

[Martin Pool (mbp)]

Yes, if you can attach that here (if you don't mind it being public) that would be great. If the data is private but you're ok to share it with a developer, talk to us on irc or email.

[Christian Reis (kiko)]

Same thing is happening to me :-(

[Christian Reis (kiko)]

kiko@anthem:/etc/network$ sudo bzr commit -m "Issue a conntrack flush when changing the routing so we don't end up with packets falling in the wrong interface" .
bzr: ERROR: Working tree is out of date, please run 'bzr update'.

kiko@anthem:/etc/network$ sudo bzr update
All changes applied successfully.
bzr: ERROR: An inconsistent delta was supplied involving 'gdm/failsafeBlacklist', 'failsafeblacklist-20101011211133-ux797y0aieyjrmnf-718'
reason: Unable to find block for this record. Was the parent added?

[Christian Reis (kiko)]

This is how I got to this point. I issued a bzr commit that removed a couple of files:

iko@anthem:/etc/network$ sudo bzr commit -m "Issue a conntrack flush when changing the routing so we don't end up with packets falling in the wrong interface"
[...]
missing gdm/failsafeBlacklist
modified gdm/failsafeBlacklist
missing gdm/failsafeXServer
modified gdm/failsafeXServer
missing gdm/failsafeXinit
modified gdm/failsafeXinit
[...]
Committed revision 308.

kiko@anthem:/etc/network$ sudo bzr uncommit
  308 Christian Reis 2011-11-18
      Issue a conntrack flush when changing the routing so we don't end up with packets falling in the wrong interface

The above revision(s) will be removed.
Uncommit these revisions? [y/n]: y
bzr: ERROR: An inconsistent delta was supplied involving 'gdm/failsafeBlacklist', 'failsafeblacklist-20101011211133-ux797y0aieyjrmnf-718'
reason: Unable to find block for this record. Was the parent added?

[Martin Pool (mbp)]

  importance high

[Johan Dahlin (jdahlin-deactivatedaccount)]

Here's a traceback, using the same repository as Christian mentioned in comment #8:

Tue 2011-12-13 10:13:36 -0200
0.044 bazaar version: 2.4.2
0.044 bzr arguments: [u'update']
0.066 looking for plugins in /home/jdahlin/.bazaar/plugins
0.067 looking for plugins in /usr/lib/python2.7/dist-packages/bzrlib/plugins
0.093 encoding stdout as sys.stdout encoding 'UTF-8'
0.123 opening working tree '/etc'
[13987] 2011-12-13 10:13:36.601 INFO: All changes applied successfully.
0.461 Not saving DirState because _changes_aborted is set.
0.461 Not saving DirState because _changes_aborted is set.
0.461 Transferred: 0kB (0.0kB/s r:0kB w:0kB)
0.485 Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 946, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 1150, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 699, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 721, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/builtins.py", line 1513, in run
    show_base=show_base)
  File "/usr/lib/python2.7/dist-packages/bzrlib/workingtree.py", line 1492, in update
    return self._update_tree(old_tip, change_reporter, revision, show_base)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 51, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/workingtree.py", line 1563, in _update_tree
    self.set_last_revision(revision)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 51, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/workingtree_4.py", line 1081, in set_last_revision
    allow_leftmost_as_ghost=True)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 51, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/workingtree_4.py", line 1108, in set_parent_ids
    allow_leftmost_as_ghost=allow_leftmost_as_ghost)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 51, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/workingtree_4.py", line 1167, in set_parent_trees
    dirstate.update_basis_by_delta(delta, rev_id)
  File "/usr/lib/python2.7/dist-packages/bzrlib/dirstate.py", line 1624, in update_basis_by_delta
    self._update_basis_apply_adds(adds)
  File "/usr/lib/python2.7/dist-packages/bzrlib/dirstate.py", line 1694, in _update_basis_apply_adds
    "Unable to find block for this record."
  File "/usr/lib/python2.7/dist-packages/bzrlib/dirstate.py", line 1665, in _raise_inv...

[Johan Dahlin (jdahlin-deactivatedaccount)]

bzr repair-workingtree --force has no effect on this bug, it still happens.

Using bzr 2.5 beta 2.

[Vincent Ladeuil (vila)]

@kiko,johan: the magic spell is: 'bzr pull -rrevid:<email address hidden>'

For an unknown reason, your working tree got out of sync with its associated branch, if you run 'bzr qlog' before doing the pull above this should be obvious.

The pull above put the branch at the revision the working tree is expecting.

It has yet to be understood how you went into this situation... any hint ?

[Mark Grandi (markgrandi)]

sorry for the late reply, but here is a zip of the branch and the server repo that I was using when all of this went down. I don't seem to be getting the InconsistentDelta error anymore, but the branch i was using around the time the error happened is still in a very weird state, it keeps saying if i do a bzr status that it found conflicts and moved the files to one place, and then found another conflict and moved it again even though there wasn't a conflict in the first place

[Marco Vittorini Orgeas (marcovorg)]

I have just experienced the same issue: I have committed some files removed and then I issued "bzr uncommit" and now I get:

bzr: ERROR: An inconsistent delta was supplied involving 'nbproject/private/private.properties', 'private.properties-20120409153545-8w5had3ubg8yp2r2-35'
reason: Unable to find block for this record. Was the parent added?

This on:
 Bazaar (bzr) 2.4.1
  Python interpreter: /usr/bin/python 2.7.2
  Python standard library: /usr/lib/python2.7
  Platform: Linux-3.0.0-17-generic-x86_64-with-Ubuntu-11.10-oneiric
  bzrlib: /usr/lib/python2.7/dist-packages/bzrlib
  Bazaar configuration: /home/marco/.bazaar
  Bazaar log file: /home/marco/.bzr.log

I can share the folder with the repo and logs if you need them.

[Mark Grandi (markgrandi)]

marco: i would zip up your branch and your repo so people can reproduce it easier, and upload them somewhere and link here

[Tom Browder (tbrowder)]

I, too, have gotten my working tree into the inconsistent state and need to get it functional again. Any solutions yet?

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 9/27/2012 6:25 PM, Tom Browder wrote:
> I, too, have gotten my working tree into the inconsistent state and
> need to get it functional again. Any solutions yet?
>

If it is just an issue with the WT, you can run 'bzr
reset-workingtree' and it should rebuild the right files. You'll lose
information about uncommitted revisions (like after a merge), or about
renames, etc.
However it will leave changes in the WT alone otherwise.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBkfDsACgkQJdeBCYSNAAMeQgCfRflYVrq6BA401Mz9eYDib6EC
gzkAoKo5nLsd1YW+bUk2kREMr9G/u2cr
=JWgt
-----END PGP SIGNATURE-----

[Tom Browder (tbrowder)]

On Thu, Sep 27, 2012 at 11:18 AM, John Arbash Meinel
<email address hidden> wrote:
...
> On 9/27/2012 6:25 PM, Tom Browder wrote:
>> I, too, have gotten my working tree into the inconsistent state and
>> need to get it functional again. Any solutions yet?
...
> If it is just an issue with the WT, you can run 'bzr
> reset-workingtree' and it should rebuild the right files. You'll lose
> information about uncommitted revisions (like after a merge), or about
> renames, etc.
> However it will leave changes in the WT alone otherwise.

Let me get this right, John. The files and directories them selves
will NOT change, just the log and other such metadata?

I can live with that, but, if the changes I've made are rolled back, I
will pull out my hair!

Best,

-Tom

[Vincent Ladeuil (vila)]

> Let me get this right, John. The files and directories them selves will NOT change, just the log and other such metadata?

Yes. What John meant is that the changes *recorded* in the dirstate (files added, revision merged if applicable, file renames, i.e. all the information 'bzr status' cannot infer by just comparing your working files with their base revision) are lost.

Your working files are *not* modified.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 9/28/2012 3:06 AM, Tom Browder wrote:
> On Thu, Sep 27, 2012 at 11:18 AM, John Arbash Meinel
> <email address hidden> wrote: ...
>> On 9/27/2012 6:25 PM, Tom Browder wrote:
>>> I, too, have gotten my working tree into the inconsistent state
>>> and need to get it functional again. Any solutions yet?
> ...
>> If it is just an issue with the WT, you can run 'bzr
>> reset-workingtree' and it should rebuild the right files. You'll
>> lose information about uncommitted revisions (like after a
>> merge), or about renames, etc. However it will leave changes in
>> the WT alone otherwise.
>
> Let me get this right, John. The files and directories them
> selves will NOT change, just the log and other such metadata?
>
> I can live with that, but, if the changes I've made are rolled
> back, I will pull out my hair!
>
> Best,
>
> -Tom
>

Correct. So if you did a 'bzr mv a b' then the directory will still be
called 'b', but bzr will now tell you that 'a is missing' and 'b is
unknown'. You can use bzr mv --after or maybe bzr mv --auto to tell
bzr about the existing renames.

(It doesn't change the content of the working tree, it just tells bzr
to reset its knowledge about the tree.)

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBlX/IACgkQJdeBCYSNAAODdACfRXmJ6QN4tZMe3PiOC8kOsL2M
lwgAoMBNQ+Im6ERc+OFRkTvDUC9KVYqv
=kE18
-----END PGP SIGNATURE-----

[Tom Browder (tbrowder)]

On Fri, Sep 28, 2012 at 3:29 AM, John Arbash Meinel
<email address hidden> wrote:
> On 9/28/2012 3:06 AM, Tom Browder wrote:
>> On Thu, Sep 27, 2012 at 11:18 AM, John Arbash Meinel
>> <email address hidden> wrote: ...
>>> On 9/27/2012 6:25 PM, Tom Browder wrote:
>>> If it is just an issue with the WT, you can run 'bzr
>>> reset-workingtree' and it should rebuild the right files. You'll

I don't have that command in bazaar 2.5 so I guess I need to upgrade.
I'm running Ubuntu 10.10 (Maverick Meercat) so I will have to
uninstall the package and install from source.

Any recommendations on the source package to use?

Best,

-Tom

[Vincent Ladeuil (vila)]

@Tom: Try 'bzr repair-workingtree' john is a bit confused between what he proposed (yeah, he wrote that command) and what landed ;)

This has landed on bzr-2.4b1 so you should have it in 2.5.x

[Tom Browder (tbrowder)]

On Thu, Sep 27, 2012 at 11:18 AM, John Arbash Meinel
<email address hidden> wrote:
> On 9/27/2012 6:25 PM, Tom Browder wrote:
>> I, too, have gotten my working tree into the inconsistent state and
>> need to get it functional again. Any solutions yet?
...
> If it is just an issue with the WT, you can run 'bzr
> reset-workingtree' and it should rebuild the right files. You'll lose

John, I've installed bzr version 2.6b2 from a source release package
but it doesn't seem to have that command.

What doi I need to do to get it?

Best regards,

-Tom

[Tom Browder (tbrowder)]

On Fri, Sep 28, 2012 at 9:13 AM, Tom Browder <email address hidden> wrote:
> On Thu, Sep 27, 2012 at 11:18 AM, John Arbash Meinel
> <email address hidden> wrote:
>> On 9/27/2012 6:25 PM, Tom Browder wrote:
>>> I, too, have gotten my working tree into the inconsistent state and
>>> need to get it functional again. Any solutions yet?
> ...
>> If it is just an issue with the WT, you can run 'bzr
>> reset-workingtree' and it should rebuild the right files. You'll lose
>
> John, I've installed bzr version 2.6b2 from a source release package
> but it doesn't seem to have that command.
>
> What doi I need to do to get it?

Never mind, the 'repair-workingtree' command is there, but it didn't help.

However, I have finally gotten rid of the error. I did this sequence:

$ bzr repair-workingtree
bzr: ERROR: The tree does not appear to be corrupt. You probably want
"bzr revert" instead. Use "--force" if you are sure you want to reset
the working tree.

$ bzr bind
$ bzr repair-workingtree
bzr: ERROR: The tree does not appear to be corrupt. You probably want
"bzr revert" instead. Use "--force" if you are sure you want to reset
the working tree.

$ bzr update
All changes applied successfully.
bzr: ERROR: An inconsistent delta was supplied involving
'mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html',
'index.html-20120921102114-n0erei2dkpu26tny-1'
reason: Unable to find block for this record. Was the parent added?

$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
bzr: ERROR: Path(s) are not versioned:
mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html

$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login
$ bzr update
All changes applied successfully.

bzr: ERROR: An inconsistent delta was supplied involving
'mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html',
'index.html-20120921102114-n0erei2dkpu26tny-1'
reason: Unable to find block for this record. Was the parent added?

$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login
$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
bzr: ERROR: Path(s) are not versioned:
mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html

$ bza mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
adding mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
- mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
$ bzr update
All changes applied successfully.
bzr: ERROR: An inconsistent delta was supplied involving
'mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html',
'index.html-20120921102114-n0erei2dkpu26tny-1'
reason: Unable to find block for this record. Was the parent added?

$ bza mydomains/domains/usafa-1965-cs24/pics/web-site/login
$ bzr revert mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
bzr: ERROR: Path(s) are not versioned:
mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html

$ bza mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
adding mydomains/domains/usafa-1965-cs24/pics/web-site/login/index.html
$ bz...

[Olivier Dony (Odoo) (odo-openerp)]

It seems the problem can be reproduce consistently by uncommitting a commit that removes the last file(s) of a directory without removing the directory itself.

I managed to restore my WT to a working status (bzr stopped complaining about it being outdated) by doing a pull -r to the revision that I failed to uncommit, and I abandoned the idea of uncommitting it. I also completely lost the uncommitted changes that were in my WT at the time of the failed uncommit (luckily for me I'm a compulsive user of bzr diff, so they were in my terminal scrollback). It seems this could lead to serious data loss, right?

Here's how I easily reproduce it on Ubuntu 12.04 with bzr 2.5.1:

$ bzr init test
Created a standalone tree (format: 2a)
$ cd test
$ mkdir a
$ touch a/b a/c
$ bzr add
adding a
adding a/b
adding a/c
$ bzr ci -m 'added a/b and a/c'
Committing to: /tmp/test/
added a
added a/b
added a/c
Committed revision 1.
$ bzr rm a/*
deleted a/c
deleted a/b
t$ bzr ci -m 'removed b and c'
Committing to: /tmp/test/
deleted a/b
deleted a/c
Committed revision 2.
$ bzr uncommit
    2 Olivier Dony 2012-10-16
      removed b and c

The above revision(s) will be removed.
Uncommit these revisions? ([y]es, [n]o): yes
bzr: ERROR: An inconsistent delta was supplied involving 'a/b', 'b-20121016165704-khjvvmax9ab4jp8p-2'
reason: Unable to find block for this record. Was the parent added?

[Olivier Dony (Odoo) (odo-openerp)]

Correcting previous comment: I can't reproduce the issue of losing the currently uncommitted changes when triggering this bug. And as I can't remember the exact sequence of commands I issued when I was trying to fix the tree, it could very well have been my mistake. So please ignore that part.

[Gerard Krol (gerard-)]

I experienced the same problem (with a moved file). I fixed it using the instructions from Tom Browder.

In short: before you perform bzr update, add the file it complains about manually:
echo test > file/bzr/complains/about.txt
bzr add file/bzr/complains/about.txt
bzr update
del file/bzr/complains/about.txt

[Alexey Kopytov (akopytov)]

I'm hitting the same bug apparently, though it didn't involve "bzr update". The corrupted branch is here: lp:~akopytov/percona-xtrabackup/compact-backups-corrupted

When running "bzr uncommit" in that branch I get:

bzr: ERROR: An inconsistent delta was supplied involving 'test/disabled/compact.sh', 'compact.sh-20111215131326-f6carhkcyy3et660-1'
reason: Unable to find block for this record. Was the parent added?

I can't recall the exact sequence that has triggered corruption in my case, it's been a long sequence of "bzr shelve", "bzr merge", "bzr unshelve" over the last few months, and eventually "bzr commit" / "bzr uncommit".

[GuilhemBichot (guilhem-bichot)]

We are seeing the same problem with MySQL branches (remove the file, commit, uncommit, error). I am using:
Bazaar (bzr) 2.5.1
  Python interpreter: /usr/bin/python 2.6.5
  Python standard library: /usr/lib/python2.6
  Platform: Linux-2.6.32-45-generic-x86_64-with-Ubuntu-10.04-lucid
  bzrlib: /usr/lib/pymodules/python2.6/bzrlib
and I can repeat the error by using the test case in
"Olivier Dony (OpenERP) (odo-openerp) wrote on 2012-10-16: "

[Brian de Alwis (slyguy)]

Given the trace in comment #10, I suspect this is the same problem that I encountered in bug #1100385. I have a fix there, and Olivier's sequence in comment #26 works fine.

(FWIW: repair-workingtree never fixed the issue for me.)