bzr crash with TooManyConcurrentRequests

[Rowan Kerr]

Got partway through a push (although the command-line counter thing never ticked past 0/0), and lost network connection. (about 20 mins in). bzr sat apparently idle for another 40 mins or so before spitting out the following when a network connection was re-established...

Read from remote host bazaar.launchpad.net: Invalid argument
bzr: ERROR: bzrlib.errors.TooManyConcurrentRequests: The medium '<bzrlib.smart.medium.SmartSSHClientMedium object at 0x84ab70c>' has reached its concurrent request limit. Be sure to finish_writing and finish_reading on the currently open request.

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 857, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 797, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 499, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 802, in run
    use_existing_dir=use_existing_dir)
  File "/usr/lib/python2.5/site-packages/bzrlib/push.py", line 172, in _show_push_branch
    stop_revision=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/branch.py", line 1782, in push
    target.unlock()
  File "/usr/lib/python2.5/site-packages/bzrlib/remote.py", line 1471, in unlock
    self._unlock(branch_token, repo_token)
  File "/usr/lib/python2.5/site-packages/bzrlib/remote.py", line 1438, in _unlock
    repo_token or '')
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 111, in call
    result, protocol = self.call_expecting_body(method, *args)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 124, in call_expecting_body
    method, args, expect_response_body=True)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 58, in _call_and_read_response
    readv_body=readv_body)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 39, in _send_request
    protocol_version)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 94, in _construct_protocol
    request = self._medium.get_request()
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/medium.py", line 610, in get_request
    return SmartClientStreamMediumRequest(self)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/medium.py", line 785, in __init__
    raise errors.TooManyConcurrentRequests(self._medium)
TooManyConcurrentRequests: The medium '<bzrlib.smart.medium.SmartSSHClientMedium object at 0x84ab70c>' has reached its concurrent request limit. Be sure to finish_writing and finish_reading on the currently open request.

bzr 1.6.1 on python 2.5.2 (linux2)
arguments: ['/usr/bin/bzr', 'push', 'lp:~launchpad-adjutants/launchpad/moin-openid']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.95.0]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

[Andrew Bennetts (spiv)]

Thanks for the report. I see the cause: yet another situation where a finally block tramples the original exception. This time it's the finally clause in BzrBranch.push. It shouldn't be hard to fix.

bzr sitting idle for 40 minutes is just normal TCP behaviour; the client has sent a request to the server and is waiting for a response, and if the connection goes away while waiting for the response then most TCP implementatons won't notice for a very long time. My initial feeling is that it's not worth trying to do something about this (e.g. add a no-op request and the ability to send them while a real request is outstanding so that the client can periodically send pings that would cause stale TCP connections to expire sooner). Feel free to open a separate bug report for that issue if it would be useful to you, though.

[Andrew Bennetts (spiv)]

I've attached a patch to bug 230902 that should address this particular bug. Possibly the bugs are duplicates, despite the different symptoms.