Import or branch fails with "assert revmeta.revnum > parent_revnum or start_empty"

[nicholas a. evans]

With bzr 1.9, bzr-svn 0.4.15, deleting the svn-cache beforehand, I will always get the following error when attempting to branch from a particular project.

File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
  return run_bzr(argv)
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
  ret = run(*run_argv)
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
  return self.run(**all_cmd_args)
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 853, in ignore_pipe
  result = func(*args, **kwargs)
File "/home/nevans/.bazaar/plugins/svn/__init__.py", line 281, in run
  keep=keep, incremental=incremental)
File "/home/nevans/.bazaar/plugins/svn/convert.py", line 202, in convert_repository
  inter.fetch(branches=existing_branches)
File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 833, in fetch
  self._fetch_switch(self.source.transport.get_svn_repos_root(), needed, pb)
File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 767, in _fetch_switch
  assert revmeta.revnum > parent_revnum or start_empty

Unfortunately, it takes about 1.5 hours of importing (even when the repository has been backed up and copied to the local filesystem) before this error occurs. Any other further debug info that would help?

[Jelmer Vernooij (jelmer)]

Is the repository public ?

can you try with the 0.5 branch ?

[nicholas a. evans (nevans)]

Sorry, it isn't public.

But I am able to try it with the 0.5 branch. Unfortunately it dies *much*
sooner in 0.5 than it did in 0.4.16.

== (nevans@nevans-laptop) [Thu Dec 04]-(18:13:39) ==
~/Private/tmp/bzr $ bzr svn-import svn+https://svn.mdlogix.com --prefix=crms crms
The svn+ syntax is deprecated, use https://svn.mdlogix.com instead.
Initialising Subversion metadata cache in /home/nevans/.bazaar/svn-cache/0487d25d-142b-0410-8fcf-b82ac621bf97
Importing branches with prefix /crms/
Using repository layout: trunk1
/ [= ] checking revisions to fetch 41/12557
> /home/nevans/.bazaar/plugins/svn/revmeta.py(739)get_lhs_parent()
-> raise MetaHistoryIncomplete()
(Pdb)

Since I'm dropped helpfully into a debugger at that point, just let me know
what would be useful to you.

Thanks!

[Jelmer Vernooij (jelmer)]

Whoops, that should never have been committed. It's been removed now. Can you check with the current 0.5 branch?

[nicholas a. evans (nevans)]

Trying with r2257 of the 0.5 branch...

== (nevans@nevans-laptop) [Tue Dec 09]-(08:55:27) ==
~/Private/tmp/bzr $ rm -rf /home/nevans/.bazaar/{svn-cache,subversion.conf} && bzr svn-import ../svnbackup-2008-11-14-1653 --prefix=crms crms
Initialising Subversion metadata cache in /home/nevans/.bazaar/svn-cache/0487d25d-142b-0410-8fcf-b82ac621bf97
Importing branches with prefix /crms/
Using repository layout: trunk1
bzr: ERROR: bzrlib.errors.NoSuchRevision: KnitPackRepository('file:///home/nevans/Private/tmp/bzr/crms/.bzr/repository/') has no revision ('svn-v3-trunk1:0487d25d-142b-0410-8fcf-b82ac621bf97:oncology%2Ftrunk:18738',)

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 853, in ignore_pipe
    result = func(*args, **kwargs)
  File "/home/nevans/.bazaar/plugins/svn/__init__.py", line 281, in run
    to_revnum=to_revnum, prefix=prefix)
  File "/home/nevans/.bazaar/plugins/svn/convert.py", line 262, in convert_repository
    inter.fetch(needed=revmetas)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 967, in fetch
    self._fetch_revisions(needed, pb, use_replay=use_replay)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 897, in _fetch_revisions
    editor = self._get_editor(revmeta, mapping)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 836, in _get_editor
    self._get_inventory(revmeta.get_lhs_parent_revid(mapping)),
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 830, in _get_inventory
    return self.target.get_inventory(revid)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1565, in get_inventory
    return self.iter_inventories([revision_id]).next()
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1583, in _iter_inventories
    for text, revision_id in self._iter_inventory_xmls(revision_ids):
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1594, in _iter_inventory_xmls
    raise errors.NoSuchRevision(self, record.key)
NoSuchRevision: KnitPackRepository('file:///home/nevans/Private/tmp/bzr/crms/.bzr/repository/') has no revision ('svn-v3-trunk1:0487d25d-142b-0410-8fcf-b82ac621bf97:oncology%2Ftrunk:18738',)

bzr 1.10 on python 2.5.2 (linux2)
arguments: ['/usr/bin/bzr', 'svn-import', '../svnbackup-2008-11-14-1653', '--prefix=crms', 'crms']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /home/nevans/.bazaar/plugins/bzrtools [1.10]
  gtk ...

[Jelmer Vernooij (jelmer)]

Thanks. This looks like an issue with a roundtripped revision in the past. Any chance you can run this again with the attached patch? That should give a somewhat better error message.

[Jelmer Vernooij (jelmer)]

Thanks. This looks like an issue with a roundtripped revision in the past. Any chance you can run this again with the attached patch? That should give a somewhat better error message.

[Michael Gratton (mjog)]

I was getting the same assertion error at the top of this bug with bzr 0.6 and bzr-svn 0.4.13 - I just upgraded to bzr 0.10 and bzr-svn 0.5-rc1 to test it out with the patch above but the problem seems to have gone away with the upgrade.

FWIW, I did roughly the following on two repos to cause them to both start tripping the original assertion:

 * bzr co <svn repo>
 * make some changes (not sure if I committed before the svn-push)
 * bzr svn-push <new svn dir, to create a branch>
 * bzr commit
 * bzr switch <original svn repo>
 * bzr update

*bang*

I think - this happened last week, so the details are a bit hazy.

[Michael Gratton (mjog)]

Some more thoughts: with the last two commands, I was expecting to switch back to the original branch that I had checked out and hence have the changes committed to the new branch wiped from my local copy. So many bzr-svn was getting confused that the local copy had a higher rev than the one one it was bound to?

Also, the "svn switch" might actually have (stupidly) been a: svn bind <original svn repo>

[Jelmer Vernooij (jelmer)]

the new svn dir you were pushing to, was that actually a subdirectory of <svn repo> ?

[Michael Gratton (mjog)]

No, but they had a common parent.

<svn repo>:
  svn+ssh:///foo/project/trunk

<new svn dir>:
  svn+ssh:///foo/project/branches/bar

Where "svn+ssh:///foo/project/trunk" and "svn+ssh:///foo/project/branches" already existed, but "svn+ssh:///foo/project/branches/bar" did not.

[nicholas a. evans (nevans)]

Ooops... I forgot to attach this error message from five days ago (using 0.5, revno 2257):

== (nevans@nevans-laptop) [Tue Dec 09]-(14:08:46) ==
~/Private/tmp/bzr $ rm -rf /home/nevans/.bazaar/{svn-cache,subversion.conf} && bzr svn-import ../svnbackup-2008-11-14-1653 --prefix=crms crms
Initialising Subversion metadata cache in /home/nevans/.bazaar/svn-cache/0487d25d-142b-0410-8fcf-b82ac621bf97
Importing branches with prefix /crms/
Using repository layout: trunk1
bzr: ERROR: Unable to find svn-v3-trunk1:0487d25d-142b-0410-8fcf-b82ac621bf97:oncology%2Ftrunk:18738 for <RevisionMetadata for revision 18741, path crms/trunk in repository '0487d25d-142b-0410-8fcf-b82ac621bf97'>: {'crms/trunk': (u'A', 'oncology/trunk', 18740)}

[nicholas a. evans (nevans)]

So here's some backstory, this repository has some strange history in it: It started life as dozens of different repositories, but was eventually merged into one single repository (with each of the original repos imported with their full history into a directory in the root). One of the merged in repos was called "oncology" and another was called "crms". And here are some revisions that look like they might specifically affect the error message above:

~/Private/tmp $ svn log -r 18703 -v file:///home/nevans/Private/tmp/svnbackup-2008-11-14-1653/
------------------------------------------------------------------------
r18703 | nevans | 2007-04-02 15:48:56 -0400 (Mon, 02 Apr 2007) | 1 line
Changed paths:
   D /crms

removing this... it's old, not the correct version, and we are about to replace it with the oncology project's trunk
------------------------------------------------------------------------

~/Private/tmp $ svn log -r 18741 -v file:///home/nevans/Private/tmp/svnbackup-2008-11-14-1653/
------------------------------------------------------------------------
r18741 | nevans | 2007-04-03 09:03:05 -0400 (Tue, 03 Apr 2007) | 2 lines
Changed paths:
   A /crms/trunk (from /oncology/trunk:18740)

Copied from the project location to the product location

[nicholas a. evans (nevans)]

FWIW, I get the exact same error after pulling the latest for 0.5 (revno 2264)

[nicholas a. evans (nevans)]

And when I try to do a plain old branch of crms/trunk (rather than svn-import), I get the following:

~/Private/tmp/bzr $ bzr branch ../svnbackup-2008-11-14-1653/crms/trunk crms-trunk
bzr: ERROR: exceptions.AttributeError: 'NoneType' object has no attribute 'uuid'

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 981, in run
    source_branch=br_from)
  File "/home/nevans/.bazaar/plugins/svn/remote.py", line 65, in sprout
    return super(SvnRemoteAccess, self).sprout(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1117, in sprout
    result_repo.fetch(source_repository, revision_id=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1021, in fetch
    find_ghosts=find_ghosts)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 953, in fetch
    mapping, find_ghosts, pb=nested_pb)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 794, in find_until
    needed = check_revid(foreign_revid, mapping, project)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 784, in check_revid
    foreign_revid, rhs_mapping = self.source.lookup_revision_id(p, project=project)
  File "/home/nevans/.bazaar/plugins/svn/repository.py", line 614, in lookup_revision_id
    return self.revmap.get_branch_revnum(revid, layout, project)
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 202, in get_branch_revnum
    for entry_revid, branch, revnum, mapping in self.actual.discover_revprop_revids(layout, last_revnum, last_checked):
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 70, in discover_revprop_revids
    for revmeta in self.repos._revmeta_provider.iter_all_revisions(layout, None, from_revnum, to_revnum):
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1037, in filter_revisions
    for kind, rev in it:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1199, in iter_all_changes
    for kind, item in browser:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 817, in next
    return self.it()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 947, in next
    ret = self._iter.next()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1026, in do
    revmeta = process_new_rev(bp, mb, revnum, paths, revprops)
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 965, in process_new_rev
    c._set_direct_lhs_parent_revmeta(revmeta)
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 281, in _set_direct_lhs_parent_revmeta
    self._direct_lhs_parent_revmeta != parent_revmeta):
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 140, in __cmp__
    (other.uui...

[Jelmer Vernooij (jelmer)]

Any chance you can try this with the 0.5 branch ? It may be fixed there.

[Jelmer Vernooij (jelmer)]

I reproduced this on another branch and that seems to be working fine now.

[nicholas a. evans (nevans)]

I just tried again (with 0.5, revno 2270) and the svn-import failed with the exact same error as before:

$ bzr svn-import ../svnbackup-2008-11-14-1653 --prefix=crms crms
Importing branches with prefix /crms/
Using repository layout: trunk1
bzr: ERROR: Unable to find svn-v3-trunk1:0487d25d-142b-0410-8fcf-b82ac621bf97:oncology%2Ftrunk:18738 for <RevisionMetadata for revision 18741, path crms/trunk in repository '0487d25d-142b-0410-8fcf-b82ac621bf97'>: {'crms/trunk': (u'A', 'oncology/trunk', 18740)}

I'll delete the cache and try the svn-import (and also simply branching trunk) again.

[nicholas a. evans (nevans)]

still get the exact same errors when deleting the cache and only branching trunk.

[nicholas a. evans (nevans)]

Actually, the branch error is slightly different than it was last time:

$ bzr branch ../svnbackup-2008-11-14-1653/crms/trunk crms-trunk
bzr: ERROR: exceptions.AssertionError:

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 981, in run
    source_branch=br_from)
  File "/home/nevans/.bazaar/plugins/svn/remote.py", line 65, in sprout
    return super(SvnRemoteAccess, self).sprout(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1117, in sprout
    result_repo.fetch(source_repository, revision_id=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1021, in fetch
    find_ghosts=find_ghosts)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 959, in fetch
    mapping, find_ghosts, pb=nested_pb)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 800, in find_until
    needed = check_revid(foreign_revid, mapping, project)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 790, in check_revid
    foreign_revid, rhs_mapping = self.source.lookup_revision_id(p, project=project)
  File "/home/nevans/.bazaar/plugins/svn/repository.py", line 614, in lookup_revision_id
    return self.revmap.get_branch_revnum(revid, layout, project)
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 202, in get_branch_revnum
    for entry_revid, branch, revnum, mapping in self.actual.discover_revprop_revids(layout, last_revnum, last_checked):
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 70, in discover_revprop_revids
    for revmeta in self.repos._revmeta_provider.iter_all_revisions(layout, None, from_revnum, to_revnum):
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1053, in filter_revisions
    for kind, rev in it:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1215, in iter_all_changes
    for kind, item in browser:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 820, in next
    return self.it()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 956, in next
    ret = self._iter.next()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1026, in do
    assert rev.branch_path == new_name
AssertionError

[Jelmer Vernooij (jelmer)]

Sorry, I still can't reproduce this so I need your help trying to figure out how to do so.

Any chance you can give this a try with bzr-svn r2312? I've made the error message a bit clearer. It should now print the branch path that didn't match as well as the revision number of the svn revision that causes the breakage. Please paste that as well as the "svn log -v" output for that particular revision.

[nicholas a. evans (nevans)]

Thanks a lot. I wish I could give you access to the repo. You'd probably have it fixed in no time. I've also included svn logs for a few other revisions that may or may not be relevant.

== (nevans@nevans-laptop) [Fri Jan 16]-(09:17:10) ==
~/Private/tmp/bzr $ bzr branch ../svnbackup-2008-11-14-1653/crms/trunk crms-trunk
bzr: ERROR: exceptions.AssertionError: Revision 35560 has invalid branch path integration_bus/branches/jruby, expected cib/branches/jruby

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 1038, in run
    source_branch=br_from)
  File "/home/nevans/.bazaar/plugins/svn/remote.py", line 65, in sprout
    return super(SvnRemoteAccess, self).sprout(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1103, in sprout
    result_repo.fetch(source_repository, revision_id=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1118, in fetch
    find_ghosts=find_ghosts)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 991, in fetch
    mapping, find_ghosts, pb=nested_pb)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 835, in find_until
    needed = check_revid(foreign_revid, mapping, project)
  File "/home/nevans/.bazaar/plugins/svn/fetch.py", line 825, in check_revid
    foreign_revid, rhs_mapping = self.source.lookup_revision_id(p, project=project)
  File "/home/nevans/.bazaar/plugins/svn/repository.py", line 633, in lookup_revision_id
    return self.revmap.get_branch_revnum(revid, layout, project)
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 202, in get_branch_revnum
    for entry_revid, branch, revnum, mapping in self.actual.discover_revprop_revids(layout, last_revnum, last_checked):
  File "/home/nevans/.bazaar/plugins/svn/revids.py", line 70, in discover_revprop_revids
    for revmeta in self.repos._revmeta_provider.iter_all_revisions(layout, None, from_revnum, to_revnum):
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1072, in filter_revisions
    for kind, rev in it:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1234, in iter_all_changes
    for kind, item in browser:
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 820, in next
    return self.it()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 973, in next
    ret = self._iter.next()
  File "/home/nevans/.bazaar/plugins/svn/revmeta.py", line 1054, in do
    raise AssertionError("Revision %d has invalid branch path %s, expected %s" % (revnum, rev.branch_path, new_name))
AssertionError: Revision 35560 has invalid branch path integration_bus/branches/jruby, expected cib/branches/jruby

bzr 1.11rc1 on python 2.5.2 (linux2)
arguments: ['/usr/bin/bzr', 'branch', '../svnbackup-2008-11-14-1653/crms/trunk', 'crms-trunk']
encoding: 'UTF-8', fsenc...

[nicholas a. evans (nevans)]

Maybe also relevant: cib@35559 did not have any branches at all (it had the branches/tags/trunk layout, but branches and tags were empty). integration_bus@HEAD does have two branches: jruby and 2008-10-06-jruby.

[Jelmer Vernooij (jelmer)]

There have been yet more fixes in this area of the code. Any chance you can try again? It would be nice to have fixed before 0.5.0 final, which I hope to release in a couple of days.

[nicholas a. evans (nevans)]

15258.772 Resizing the inventory entry cache from 10240 to 26358
[19052] 2009-02-01 18:56:17.661 INFO: Branched 15896 revision(s).
15401.000 return code 0

Yes! Victory! Thanks. One of those last 87 commits must have fixed it. :-)

[Jelmer Vernooij (jelmer)]

Thanks for confirming!