TooManyConcurrentRequests error from abort_write_group hides underlying error

[Martin Pool]

I think my http connection closed, and I got this traceback. I think it's quite unlikely that abort_write_group can succeed over an ssh connection as it's highly likely the connection is already in use.

Some things we could do here:

 * like for unlock, have abort_write_group transform errors into warnings if something goes wrong; this is hiding information from the caller but the worst that should happen is that there will be accumulated garbage. (actually we would have to be careful as to which errors are caught this way.)

 * from the except block, log the exception that caused the write group to be aborted; or pass it in to the abort_write_group method so it can be printed later

 * have the ssh transport not fail if it's already in use but instead open another connection - this won't totally fix the problem if eg the link is down and it's not possible to open a new connection

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 913, in run
    use_existing_dir=use_existing_dir)
  File "/usr/lib/python2.5/site-packages/bzrlib/push.py", line 105, in _show_push_branch
    revision_id=revision_id, stacked_on=stacked_on)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 221, in clone_on_transport
    result_repo.fetch(local_repo, revision_id=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 989, in fetch
    find_ghosts=find_ghosts)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 2811, in fetch
    pb, find_ghosts)
  File "/usr/lib/python2.5/site-packages/bzrlib/fetch.py", line 112, in __init__
    self.to_repository.abort_write_group()
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 524, in abort_write_group
    self._abort_write_group()
  File "/usr/lib/python2.5/site-packages/bzrlib/repofmt/pack_repo.py", line 1842, in _abort_write_group
    self._pack_collection._abort_write_group()
  File "/usr/lib/python2.5/site-packages/bzrlib/repofmt/pack_repo.py", line 1735, in _abort_write_group
    self._new_pack.abort()
  File "/usr/lib/python2.5/site-packages/bzrlib/repofmt/pack_repo.py", line 325, in abort
    self.upload_transport.delete(self.random_name)
  File "/usr/lib/python2.5/site-packages/bzrlib/transport/remote.py", line 292, in delete
    resp = self._call2('delete', self._remote_path(relpath))
  File "/usr/lib/python2.5/site-packages/bzrlib/transport/remote.py", line 170, in _call2
    return self._client.call(method, *args)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 122, in call
    result, protocol = self.call_expecting_body(method, *args)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 135, in call_expecting_body
    method, args, expect_response_body=True)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 69, in _call_and_read_response
    readv_body=readv_body)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 42, in _send_request
    protocol_version)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/client.py", line 105, in _construct_protocol
    request = self._medium.get_request()
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/medium.py", line 672, in get_request
    return SmartClientStreamMediumRequest(self)
  File "/usr/lib/python2.5/site-packages/bzrlib/smart/medium.py", line 870, in __init__
    raise errors.TooManyConcurrentRequests(self._medium)
TooManyConcurrentRequests: The medium '<bzrlib.smart.medium.SmartSSHClientMedium object at 0xa60280c>' has reached its concurrent request limit. Be sure to finish_writing and finish_reading on the currently open request.

[Jonathan Lange (jml)]

Possibly related to bugs 243391, 246233, 273177 and 230902 (the last one is less likely, but bug 273177 seems to indicate they might be dupes).

[Andrew Bennetts (spiv)]

Patch sent to list.

[Andrew Bennetts (spiv)]

This case now produces errors like:

$ ./bzr push bzr://...
bzr: ERROR (ignored): The medium '<bzrlib.smart.medium.SmartTCPClientMedium object at 0x914a0cc>' has reached its concurrent request limit. Be sure to finish_writing and finish_reading on the currently open request.
bzr: ERROR: Connection closed: please check connectivity and permissions (and try -Dhpss if further diagnosis is required)

Not perfect, but much saner.

[Andrew Bennetts (spiv)]

This fix is in bzr.dev and 1.10rc1.