race condition when doing a bzr commit to an svn repo

[Philip Peitsch]

If a bzr checkin is large enough and takes a few minutes to complete, there is a window where another check-in can be made, and bzr doesn't realise it has happened.

The basic setup is:
1) Create a svn branch in an svn repo
2) Create a bzr co and an svn co
3) Add a single file into the svn checkout and prepare this for committing (svn add the file, but don't commit yet)
4) Add a huge number of files in the bzr checkout, and prepare this for committing (bzr add all the files)

5) Start the bzr commit happening (bzr commit -m "some stuff"). When this starts uploading the data to the master, and while this is processing, commit the svn copy (svn commit -m "some other stuff")

6) run a bzr log and svn log. The svn repo will show 3 revisions, and the bzr log will only show 2. The bzr checkout will never pick up the changes made in the svn repo (i.e., the new file added to the svn checkout)

I've attached a script that in theory will reproduce this on demand. On the script, when it hits the end, it will print out an svn log and a bzr log. On my machine this shows:

------------------------------------------------------------------------
r3 | zpp | 2010-04-30 13:29:02 +1000 (Fri, 30 Apr 2010) | 1 line

Committing bzr changes
------------------------------------------------------------------------
r2 | zpp | 2010-04-30 13:28:58 +1000 (Fri, 30 Apr 2010) | 1 line

Adding random file
------------------------------------------------------------------------
r1 | zpp | 2010-04-30 13:28:55 +1000 (Fri, 30 Apr 2010) | 1 line

Initial import
------------------------------------------------------------------------

/tmp/bzr-svn-race-test ~
------------------------------------------------------------
revno: 2
committer: zpp <zpp@launchpad>
branch nick: testproj
timestamp: Fri 2010-04-30 13:28:57 +1000
message:
  Committing bzr changes
------------------------------------------------------------
revno: 1
svn revno: 1 (on /testproj)
committer: zpp
timestamp: Fri 2010-04-30 03:28:55 +0000
message:
  Initial import
~

To hack around the problem, it seems that I need to change the bzr:base-revision property on the offending bzr commit (#3 in the example), so that it is based on rev2, not rev1... of course, if there are conflicting changes made between rev2 & rev3, this is not a good idea!

I suspect there is some way to repair these through creation of patches in svn... but am not quite sure yet. Will post back if I figure it out.

I can manually set the base revision by doing: +++=== DO NOT USE THIS IN A PRODUCTION ENV ===+++
# enable revprop changing in the repo
echo "#!/bin/bash" > //tmp/bzr-svn-race-test/svnrepo/hooks/pre-revprop-change
echo "exit 0" >> /tmp/bzr-svn-race-test/svnrepo/hooks/pre-revprop-change
chmod +x /tmp/bzr-svn-race-test/svnrepo/hooks/pre-revprop-change

svn propget --revprop -r3 'bzr:base-revision'
svn propset --revprop -r3 'bzr:base-revision' svn-v4:<uuid from previous result>:testproj:2
rm -rf ~/.cache/bazaar # clear bzr's svn cache as it is still stale
bzr co file:///tmp/bzr-svn-race-test/svnrepo/testproj bzr-co-fixed
bzr log bzr-co-fixed

[Philip Peitsch (philip-peitsch)]

So to add some more information, once this problem occurs in the repository, it becomes impossible to pull up to a certain revision (e.g., bzr pull -r 10). bzr-svn will assert in /usr/lib/python2.6/dist-packages/bzrlib/plugins/svn/branch.py in the get_rev_id method.

[Philip Peitsch (philip-peitsch)]

I've written an svn pre-commit hook that can detect and reject these overlapping commits.

Basically, it checks each incoming commit. If it is a bzr commit, the hook will ensure that the base-revision is either pointing to the previous svn checkin (it expects the part after the : in the base-revision to be this id), or matches the previous checkin bzr:revision-id if it exists.

Not entirely pretty, but seems to work well enough until this is fixed in bzr-svn directly.

[Philip Peitsch (philip-peitsch)]

Here is the text for a modified version of get_rev_id to help detect these issues:

def get_rev_id(self, revno, history=None):
        """Find the revision id of the specified revno."""
        if revno == 0:
            return NULL_REVISION
        last_revno = self.revno()
        if revno <= 0 or revno > last_revno:
            raise NoSuchRevision(self, revno)
        count = last_revno - revno
        previous_revno = 0
        previous_foreign_revno = 0
        print "Count:", count, " LastRev:", last_revno, " RevNo:", revno
        for (revmeta, mapping) in self._revision_meta_history():
            cur_revno = revmeta.get_revno(mapping)
            if previous_revno > 0 and not previous_revno - cur_revno == 1:
                print "%s->%s"%(revmeta.get_foreign_revid()[2],revmeta.get_revno(mapping)),
                print "%s->%s"%(previous_foreign_revno,previous_revno),
                print count
            if revmeta.is_hidden(mapping):
                print "Hidden revision",
                continue
            if count == 0:
                if not revmeta.get_revno(mapping) == revno:
                    print "###", last_revno, revmeta.get_revno(mapping), revno
                assert revmeta.get_revno(mapping) == revno
                return revmeta.get_revision_id(mapping)
            count -= 1
            previous_revno = cur_revno
            previous_foreign_revno = revmeta.get_foreign_revid()[2]
        raise AssertionError