KeyError in remove_recursive_id during commit

Bug #494221 reported by Peter Hardman on 2009-12-08
84
This bug affects 11 people
Affects Status Importance Assigned to Milestone
Bazaar
High
Vincent Ladeuil
2.0
High
Vincent Ladeuil
2.1
High
Vincent Ladeuil
bzr (Ubuntu)
Undecided
Unassigned
Lucid
Undecided
Unassigned

Bug Description

Traceback:
 Traceback (most recent call last):
   File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
     return the_callable(*args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 1037, in run_bzr
     ret = run(*run_argv)
   File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
     return self.run(**all_cmd_args)
   File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 3058, in run
     exclude=safe_relpath_files(tree, exclude))
   File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 197, in commit
     result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 229, in commit
     *args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/commit.py", line 401, in commit
     self.work_tree.unversion(self.deleted_ids)
   File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 53, in tree_write_locked
     return unbound(self, *args, **kwargs)
   File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1239, in unversion
     self._inventory.remove_recursive_id(file_id)
   File "/usr/lib/python2.5/site-packages/bzrlib/inventory.py", line 1424, in remove_recursive_id
     to_find_delete = [self._byid[file_id]]
 KeyError: 'flocktemplateshtml-20091003114949-z0z1gwp6y7iwfxtm-1'

Related branches

Martin Pool (mbp) on 2009-12-08
summary: - Bazaar crashed performing commit with command-line message to a branch.
+ KeyError in remove_recursive_id during commit
description: updated

Same problem here after deletion of some images (using rm .*png) and adding some new files (bzr add):

bzr: ERROR: exceptions.KeyError: 'add_func_en.png-20091219153325-gsid8xpcjd8bbp7z-150'

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 3058, in run
    exclude=safe_relpath_files(tree, exclude))
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 197, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 229, in commit
    *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commit.py", line 401, in commit
    self.work_tree.unversion(self.deleted_ids)
  File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 53, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1239, in unversion
    self._inventory.remove_recursive_id(file_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/inventory.py", line 1424, in remove_recursive_id
    to_find_delete = [self._byid[file_id]]
KeyError: 'add_func_en.png-20091219153325-gsid8xpcjd8bbp7z-150'

bzr 2.0.2 on python 2.5.4 (Linux-2.6.30-2-amd64-x86_64-with-debian-squeeze-sid)
arguments: ['/usr/bin/bzr', 'commit', '-F', 'change.log']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'de_DE.UTF-8'
plugins:
  bzr_difftools /home/user/.bazaar/plugins/bzr_difftools [unknown]
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [2.0.1]
  dbus /usr/lib/python2.5/site-packages/bzrlib/plugins/dbus [0.1.0dev]
  extmerge /home/user/.bazaar/plugins/extmerge [unknown]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.97.0.final]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [2.0.2]
  loom /usr/lib/python2.5/site-packages/bzrlib/plugins/loom [1.4.0dev]
  netrc_credential_store /usr/lib/python2.5/site-packages/bzrlib/plugins/netrc_credential_store [2.0.2]
  search /usr/lib/python2.5/site-packages/bzrlib/plugins/search [1.7.0dev]
  stats /usr/lib/python2.5/site-packages/bzrlib/plugins/stats [unknown]
  upload /usr/lib/python2.5/site-packages/bzrlib/plugins/upload [1.0.0dev]

Martin Pool (mbp) on 2009-12-21
Changed in bzr:
status: New → Confirmed
importance: Undecided → High

Once again, using bzr 2.0.3 on Debian testing.
Cannot reproduce this error.
log attached.

2010/1/17 Thomas 'thorn' Hornik <email address hidden>:
> Once again, using bzr 2.0.3 on Debian testing.
> Cannot reproduce this error.
> log attached.

Do you mean that you think it's fixed in 2.0.3, or it's just hard to
reproduce consistently in any version?
--
Martin <http://launchpad.net/~mbp/>

> Do you mean that you think it's fixed in 2.0.3, or it's just hard to
> reproduce consistently in any version?

This bug appeared (randomly?) both in 2.0.2 and 2.0.3. Most the time there are no problems committing changes (even with removed files). But sometimes this bug appears.

Dario Bertini (berdario) wrote :

same crash here

ubuntu amd64 with bzr 2.0.2 and an alias set to "commit --strict"

removed a folder of files, the commit went fine for a handful of them... but on another one it crashes (i'll attach the crashlog)

after the crash it says that the working tree is out of date (i haven't yet tried to see the log, update the tree or retry the commit)

judging from the stacktrace it seems this bug is quite common... shouldn't some of these (or this one) be marked as duplicates?

https://bugs.launchpad.net/bzr/+bug/435000
https://bugs.launchpad.net/bzr/+bug/494221
https://bugs.launchpad.net/bzr/+bug/391580

Martin Pool (mbp) wrote :

On 23 February 2010 07:36, Dario Bertini <email address hidden> wrote:
> same crash here
>
> ubuntu amd64 with bzr 2.0.2 and an alias set to "commit --strict"
>
> removed a folder of files, the commit went fine for a handful of them...
> but on another one it crashes (i'll attach the crashlog)
>
> after the crash it says that the working tree is out of date (i haven't
> yet tried to see the log, update the tree or retry the commit)
>
> judging from the stacktrace it seems this bug is quite common...
> shouldn't some of these (or this one) be marked as duplicates?
>
> https://bugs.launchpad.net/bzr/+bug/435000
> https://bugs.launchpad.net/bzr/+bug/494221
> https://bugs.launchpad.net/bzr/+bug/391580

Yes, thanks for finding them.

--
Martin <http://launchpad.net/~mbp/>

tags: added: commit inventory
Frits Jalvingh (fjalvingh) wrote :

This is not exactly building confidence- bzr now aborts in this way for *every* remove of a tree structure. I added a trivial script to illustrate the problem:
jal@odeon:~$ ./trivial
Created a standalone tree (format: 2a)
adding a
adding a/b
adding a/b/c
adding a/b/c/d
adding a/b/c/d/file
Committing to: /home/jal/test/a/
added a
added a/b
added a/b/c
added a/b/c/d
added a/b/c/d/file
Missing checkcommit.jar, not checking commits.
Committed revision 1.
Committing to: /home/jal/test/a/
missing a/b
modified a/b
missing a/b/c
modified a/b/c
missing a/b/c/d
modified a/b/c/d
missing a/b/c/d/file
modified a/b/c/d/file
Missing checkcommit.jar, not checking commits.
bzr: ERROR: exceptions.KeyError: 'c-20100223092848-lv5tl8m8u8au91dd-3'

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    attaching the crash file
        /home/jal/.cache/crash/bzr-20100223092848-27410.crash
    and including a description of the problem.

    The crash file is plain text and you can inspect or edit it to remove
    private information.

I've just tried Fritz Jahlvingh's test script on my Ubuntu 8.04 amd64 machine :

Bazaar (bzr) 2.0.4
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib/python2.5
  Platform: Linux-2.6.24-27-generic-x86_64-with-debian-lenny-sid
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib
  Bazaar configuration: /home/home/.bazaar
  Bazaar log file: /home/home/.bzr.log

Sure enough it fails as advertised:

bzr: ERROR: exceptions.KeyError: 'c-20100223095810-vsbchpz1r6kuxe3x-3'

But If I alter line 11:

- rm -r a/b
+ bzr rm a/b

then i no longer get the bug.

FWIW

Pete

Feeling brave today.
I just installed the latest update from Canonical:

Bazaar (bzr) 2.1.0
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib/python2.5
  Platform: Linux-2.6.24-27-generic-x86_64-with-debian-lenny-sid
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib
  Bazaar configuration: /home/home/.bazaar
  Bazaar log file: /home/home/.bzr.log

Same behaviour as above. Using rm causes a crash, using bzr rm is OK.

Pete

John A Meinel (jameinel) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frits Jalvingh wrote:
> This is not exactly building confidence- bzr now aborts in this way for *every* remove of a tree structure. I added a trivial script to illustrate the problem:
> jal@odeon:~$ ./trivial
> Created a standalone tree (format: 2a)
> adding a
> adding a/b
> adding a/b/c
> adding a/b/c/d
> adding a/b/c/d/file
> Committing to: /home/jal/test/a/
> added a
> added a/b
> added a/b/c
> added a/b/c/d
> added a/b/c/d/file
> Missing checkcommit.jar, not checking commits.
> Committed revision 1.
> Committing to: /home/jal/test/a/
> missing a/b
> modified a/b
> missing a/b/c
> modified a/b/c
> missing a/b/c/d
> modified a/b/c/d
> missing a/b/c/d/file
> modified a/b/c/d/file
> Missing checkcommit.jar, not checking commits.
> bzr: ERROR: exceptions.KeyError: 'c-20100223092848-lv5tl8m8u8au91dd-3'
>

I've been seeing it more as well. just doing 'bzr rm' before 'bzr
commit' works fine. Also, the commit does succeed, and 'bzr update' then
brings you into the correct state.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkuH5D8ACgkQJdeBCYSNAAN9WwCgl3ghBq3x9jNS9A/DgRaE8aVk
Ox4AoIpja5SxFbI8KhVZrQeTbJCUagAG
=hkJG
-----END PGP SIGNATURE-----

Frits Jalvingh (fjalvingh) wrote :

By itself when the bug occurs simply doing a bzr update usually is all that is needed to get things working again.

My point is more that regardless of ways around this, having such a simple operation abort in this way is not exactly confidence building in the product.

Martin Pool (mbp) on 2010-06-07
Changed in bzr:
milestone: none → 2.2.0
Marius Kruger (amanica) wrote :

the attached script does not crash bzr for me (using bzr.dev)

Vincent Ladeuil (vila) on 2010-07-12
Changed in bzr:
assignee: nobody → Vincent Ladeuil (vila)
Vincent Ladeuil (vila) on 2010-07-13
Changed in bzr:
status: Confirmed → In Progress
Vincent Ladeuil (vila) on 2010-07-14
Changed in bzr:
status: In Progress → Fix Released
Martin Pool (mbp) wrote :

Bug 705802 shows people are still hitting this in Lucid, so we could possibly use it to drive an SRU.

Vincent Ladeuil (vila) on 2011-01-24
tags: added: sru

Accepted bzr into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in bzr (Ubuntu Lucid):
status: New → Fix Committed
tags: added: verification-needed
Max Bowsher (maxb) wrote :

Per the guidelines attached to the bzr MicroReleaseException, I have installed the package from lucid-proposed in a chroot, and run the full testsuite - passed successfully.

Martin Pitt (pitti) wrote :

Assuming this is fixed in maverick and above, closing floating task.

Changed in bzr (Ubuntu):
status: New → Fix Released
tags: added: verification-done
removed: verification-needed
Martin Pitt (pitti) wrote :

Thanks Max!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package bzr - 2.1.4-0ubuntu1

---------------
bzr (2.1.4-0ubuntu1) lucid-proposed; urgency=low

  * Update watch file to use 2.1 series.
  * New upstream release.
   + Fix file descriptors leaks in dirstate compiled extension. LP: #583486
   + Refuse to stack on repositories in incompatible formats. LP: #562380
   + Don't delete nested trees/repos. LP: #572098
   + Fix 'bzr switch' crash when a 'ConfigurableFileMerger' is used. LP: #559436
   + Fix compatibility with older smart servers. LP: #528041
   + Fix symlinks addition. LP: #192859
   + Properly unversion children of unversioned directories. LP: #494221
   + Lock configuration files in '~/.bazaar' for updates. LP: #525571
   + Fix 'bzr commit <symlink>'. LP: #128562
   + Fix `lp:` urls when behind an http proxy. LP: #558343
   + Stop using edge.launchpad.net. LP: #583667
 -- Max Bowsher <email address hidden> Tue, 17 May 2011 09:54:17 +0100

Changed in bzr (Ubuntu Lucid):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers