KeyError on branch large stacked repo

[Andrew West]

Trying to branch a external svn repository gives the following error.

andy@hostname:~/bzrsvntest/svnbranch$ bzr branch --stacked svn://andy@127.0.0.1/svnbranch/branches/V3 svnbranch
using <bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack5RichRoot object at 0x894b3ac> for stacking
bzr: ERROR: exceptions.KeyError: 'tools/showimg2/debug.h'
Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 857, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 797, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 499, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 866, in run
    hardlink=hardlink, stacked=stacked)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/remote.py", line 64, in sprout
    return super(SvnRemoteAccess, self).sprout(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1114, in sprout
    hardlink=hardlink)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1410, in create_workingtree
    accelerator_tree=accelerator_tree, hardlink=hardlink)
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1369, in initialize
    basis = branch.repository.revision_tree(revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1663, in revision_tree
    inv = self.get_revision_inventory(revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1627, in get_revision_inventory
    return self.get_inventory(revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1520, in get_inventory
    return self.iter_inventories([revision_id]).next()
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1538, in _iter_inventories
    for text, revision_id in self._iter_inventory_xmls(revision_ids):
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1545, in _iter_inventory_xmls
    for record in stream:
  File "/usr/lib/python2.5/site-packages/bzrlib/knit.py", line 1221, in get_record_stream
    needed_from_fallback - absent_keys)
  File "/usr/lib/python2.5/site-packages/bzrlib/knit.py", line 1006, in _get_content_maps
    'unordered', True):
  File "/usr/lib/python2.5/site-packages/bzrlib/versionedfile.py", line 1378, in get_record_stream
    lines = self._get_lines(k)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/versionedfiles.py", line 98, in get_lines
    return osutils.split_lines(self.repository.get_inventory_xml(key))
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/repository.py", line 651, in get_inventory_xml
    return self.serialise_inventory(self.get_inventory(revision_id))
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/repository.py", line 396, in get_inventory
    return self.revision_tree(revision_id).inventory
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/repository.py", line 508, in revision_tree
    return SvnRevisionTree(self, revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/tree.py", line 77, in __init__
    mapping)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/repository.py", line 399, in get_fileid_map
    return self.fileid_map.get_map(self.uuid, revnum, path, mapping)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/fileids.py", line 294, in get_map
    self.actual.update_map(map, revid, idmap, changes)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/svn/fileids.py", line 191, in update_map
    delta[p] = map[p][0]

'tools/showimg2/debug.h' exists in the sqlite database cache-v4 changed_path table with the following line

122423|M|svnbranch/branches/V3/tools/showimg2/debug.h||-1

Looks like revno 122423 was pulled in from somewhere else, querying cache-v4 for revno 122324 gives

122423|A|svnbranch/branches/V3/tools/showimg2|svnbranch/branches/V2/usr/local/delta/src|120883
122423|M|svnbranch/branches/V3/tools/showimg2/Makefile||-1
122423|D|svnbranch/branches/V3/tools/showimg2/TwDriver.h||-1
122423|M|svnbranch/branches/V3/tools/showimg2/args.h||-1
122423|M|svnbranch/branches/V3/tools/showimg2/debug.h||-1
122423|M|svnbranch/branches/V3/tools/showimg2/main.cpp||-1
122423|A|svnbranch/branches/V3/tools/showimg2/showimg2_xwin|svnbranch/branches/V3/tools/showimg/showimg_xwin|122034
122423|D|svnbranch/branches/V3/tools/showimg2/svgalib||-1
122423|D|svnbranch/branches/V3/tools/showimg2/tars/SDL-1.2.13.tar.gz||-1
122423|M|svnbranch/branches/V3/tools/showimg2/test-showimg||-1

[Jelmer Vernooij (jelmer)]

Does a normal fetch work ok?

[Robert Boggs (rab-isotek)]

I think I have hit this bug too during normal branching from several of our subversion repositories. (See example traceback attached)

The bug seems to have cropped up recently. I have another machine with an older bzr set up which makes the branches fine. That is configured with bzr 1.5, bzr-svn 0.4.10, rebase 0.3.0, bzr-gtk 0.94.0 and bzrtools 1.5.0 running on standard Fedora 8 with subversion 1.4.4-7 fedora srpm code patched as advised. I am using a shared bzr repo in rich-root-pack format.

The failing machine is running standard fedora 8 + patched subversion as the other, but bzr 1.6, bzrtools 1.6.0, bzr-gtk 0.95.0, rebase 0.4.0dev0 and bzr-svn 0.4.12. Shared bzr repo in 1.6-rich-root format.

Digging through the history for the file mentioned in each failure, most of them have been merged or copied to their current location from branches that have subsequently been deleted. There were only two exceptions. In one case the file had been svn copied from another part of the same build tree (not another branch) and the original and its enclosing directory were later deleted. In the other the file had been created with "svn copy" in the working directory and then modified before being committed, so the changeset shows an add with changes. The original file still exists in the head of the subversion source. There are no further change sets that affect the new file.

I hope this helps.

[Jelmer Vernooij (jelmer)]

Any chance you can post the output of "svn log -v" on revnum 122423 as well as svn log -v on the particular file that was failing ?

[Andrew West (andrewwest)]

svn log svn://andy@127.0.0.1/svnbranch/branches/V3 -r 122423
Changed paths:
   A /svnbranch/branches/V3/tools/showimg2 (from /target/branches/V25_10/usr/local/delta/src:120883)
   M /svnbranch/branches/V3/tools/showimg2/Makefile
   D /svnbranch/branches/V3/tools/showimg2/TwDriver.h
   M /svnbranch/branches/V3/tools/showimg2/args.h
   M /svnbranch/branches/V3/tools/showimg2/debug.h
   M /svnbranch/branches/V3/tools/showimg2/main.cpp
   A /svnbranch/branches/V3/tools/showimg2/showimg2_xwin (from /svnbranch/branches/V3/tools/showimg/showimg_xwin:122034)
   D /svnbranch/branches/V3/tools/showimg2/svgalib
   D /svnbranch/branches/V3/tools/showimg2/tars/SDL-1.2.13.tar.gz
   M /svnbranch/branches/V3/tools/showimg2/test-showimg

[Jelmer Vernooij (jelmer)]

This is *probably* a dupe of bug 277043.

[Jelmer Vernooij (jelmer)]

This should work ok in the 0.5 branch.