"database is locked" bzr internal error

[Rexbard]

Attempted to check out an svn repository using a command in the form:
    bzr co svn+http://$SVNREPOS/trunk svn-via-bzr-svn-trunk

$SVNREPOS represents an existing svn repository that can be used with "svn co $SVNREPOS/trunk"
The checkout was performed in a regular directory (no init-repo).
There is a proxy server in between.
The copying of the SVN revisions completed.
The analysis of the SVN revisions probably completed (wasn't watching closely).
The proxy server may have went down (it has been up and down all day for very short periods (< 1 min).
A "bzr branch" of the svn repository was started elsewhere before this error occurred.
   This other branch attempt also failed, due to attempting to perform an svn+http
   branch into the bzr 1.0 default database format, "knitpack". The reported error was:
       bzr: ERROR: Repository KnitPackRepository('file:///home/john/bzr-repo/.bzr/repository/')
       is not compatible with repository SvnRepository('svn+http://$SVNREPOS')

$ bzr selftest svn
testing: /usr/bin/bzr
   /usr/lib/python2.4/site-packages/bzrlib (1.1.0.dev.0 python2.4.3.final.0)

XFAIL: bzrlib.plugins.svn.tests.test_push.PushNewBranchTests.test_push_overwrite
push --overwrite not supported yet

[689/720 in 980s, 1 known failures, 9 skipped] bzrlib.plugins.svn.tests.test_workingtree.TestWorkingTree.test_lock_write /usr/lib/python2.4/site-packages/bzrlib/lockable_files.py:110: UserWarning: file group LockableFiles(<bzrlib.transport.local.LocalTransport url=file:///tmp/testbzr-TbHAe1.tmp/tmpN51faw/work/dc/.svn/bzr/>) was not explicitly unlocked
  warn("file group %r was not explicitly unlocked" % self)
[690/720 in 981s, 1 known failures, 9 skipped] bzrlib.plugins.svn.tests.test_workingtree.TestWorkingTree.test_move /usr/lib/python2.4/site-packages/bzrlib/lockable_files.py:110: UserWarning: file group LockableFiles(<bzrlib.transport.local.LocalTransport url=file:///tmp/testbzr-TbHAe1.tmp/tmpzKp2Yh/work/dc/.svn/bzr/>) was not explicitly unlocked
  warn("file group %r was not explicitly unlocked" % self)
Exception bzrlib.errors.LockBroken: <bzrlib.errors.LockBroken instance at 0x16569f2c> in <bound method LockableFiles.__del__ of LockableFiles(<bzrlib.transport.local.LocalTransport url=file:///tmp/testbzr-TbHAe1.tmp/tmpzKp2Yh/work/dc/.svn/bzr/>)> ignored
XFAIL: bzrlib.plugins.svn.tests.test_workingtree.TestWorkingTree.test_revert
revert not supported yet

[720/720 in 1036s, 2 known failures, 9 skipped] bzrlib.plugins.svn.tests.test_blackbox.TestBranch.test_log_empty
----------------------------------------------------------------------
Ran 720 tests in 1038.714s

OK (known_failures=2)
9 tests skipped
tests passed

$ bzr co svn+http://$SVNREPOS/trunk svn-via-bzr-svn-trunk
bzr: ERROR: pysqlite2.dbapi2.OperationalError: database is locked

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 806, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 762, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.4/site-packages/bzrlib/builtins.py", line 989, in run
    accelerator_tree)
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 192, in create_checkout
    return self._create_heavyweight_checkout(to_location, revision_id)
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 145, in _create_heavyweight_checkout
    checkout_branch.pull(self, stop_revision=revision_id)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1731, in pull
    run_hooks=run_hooks)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1530, in pull
    self.update_revisions(source, stop_revision, overwrite=overwrite)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1469, in update_revisions
    other_last_revno, other_last_revision = other.last_revision_info()
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 233, in last_revision_info
    return self.revision_id_to_revno(last_revid), last_revid
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 246, in revision_id_to_revno
    history = self.revision_history()
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 267, in revision_history
    self._revision_history = self._generate_revision_history(last_revnum)
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/branch.py", line 207, in _generate_revision_history
    str(self.scheme)))
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/repository.py", line 700, in generate_revision_id
    revid = self.revmap.lookup_branch_revnum(revnum, path, scheme)
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/svn/revids.py", line 152, in lookup_branch_revnum
    revid = self.cachedb.execute(
OperationalError: database is locked

bzr 1.1.0.dev.0 on python 2.4.3.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'co', 'svn+http://$SVNREPOS/trunk', 'svn-via-bzr-svn-trunk']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.4/site-packages/bzrlib/plugins/bzrtools [1.0.0]
  gtk /usr/lib/python2.4/site-packages/bzrlib/plugins/gtk [0.93.0]
  launchpad /usr/lib/python2.4/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.4/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
  svn /usr/lib/python2.4/site-packages/bzrlib/plugins/svn [0.4.6dev0]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

[James Westby (james-w)]

Hi,

This appears to be a bzr-svn bug. I have transferred it to that.

Thanks,

James

[Jelmer Vernooij (jelmer)]

This would be caused by two instances of bzr trying to use the same Subversion repository at the same time.

[Wesley J. Landaker (wjl)]

I just ran into this bug myself, probably for the same reason. It would be nice if the database wasn't locked for the entire bzr operation (like a really long branch or checkout) but instead was only locked for the duration of a transaction. I don't know enough about what kind of things are stored in the database to know if this is feasible in the way bzr-svn is using it, but sqlite itself certainly supports this kind of operation.

But, if it's really a permanent limitation of bzr-svn that it can't ever have two instances using the same repository, I'd suggest that the exception should be caught and a more reasonable error message returned. As is, it is a crash and tells the user to report a bug.

Here is the traceback I got, just in case it is informational:

$ bzr co svn+https://...
bzr: ERROR: pysqlite2.dbapi2.OperationalError: database is locked

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 834, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 790, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.4/site-packages/bzrlib/builtins.py", line 1012, in run
    accelerator_tree, hardlink)
  File "/home/wjlanda/.bazaar/plugins/svn/branch.py", line 200, in create_checkout
    return self._create_heavyweight_checkout(to_location, revision_id, hardlink=hardlink)
  File "/home/wjlanda/.bazaar/plugins/svn/branch.py", line 153, in _create_heavyweight_checkout
    checkout_branch.pull(self, stop_revision=revision_id)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1714, in pull
    run_hooks=run_hooks)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1513, in pull
    self.update_revisions(source, stop_revision, overwrite=overwrite)
  File "/usr/lib/python2.4/site-packages/bzrlib/decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.4/site-packages/bzrlib/branch.py", line 1452, in update_revisions
    other_last_revno, other_last_revision = other.last_revision_info()
  File "/home/wjlanda/.bazaar/plugins/svn/branch.py", line 232, in last_revision_info
    last_revid = self.last_revision()
  File "/home/wjlanda/.bazaar/plugins/svn/branch.py", line 264, in last_revision
    return self.repository.generate_revision_id(last_revnum, self.get_branch_path(last_revnum), self.mapping)
  File "/home/wjlanda/.bazaar/plugins/svn/repository.py", line 578, in generate_revision_id
    str(mapping.scheme))
  File "/home/wjlanda/.bazaar/plugins/svn/revids.py", line 118, in insert_revid
    (min_revnum, max_revnum, revid, branch, scheme))
OperationalError: database is locked

bzr 1.3 on python 2.4.5.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'co', 'svn+https://......

[Jelmer Vernooij (jelmer)]

bzr-svn probably needs a new database format to properly handle this.

[Jelmer Vernooij (jelmer)]

a new database format has landed based on TDB, which supports multiple writers.

[Marius Kruger (amanica)]

I got into this mess, so I eventually figured out there are processes running which is actually holding the lock.
With the following command I was able to find them:
lsof |grep svn |less
then I killed them and I could work with my branches again.

The proper fix seems to be to use tdb:
[15:46:59] <jelmer> AmanicA: I pushed a change to bzr-svn 0.6 the other day that allows it to use tdb rather than sqlite
[15:47:16] <jelmer> AmanicA: tdb allows multiple writers (i.e. no more locking madness)
[15:47:29] <AmanicA> how do I activate that?
[15:47:46] <jelmer> AmanicA: install python-tdb