crash in 'bzr pull' when the network connection breaks

[hackworks]

- I had done a 'bzr pull' and since there is no information about progress and slow performance on emacs repo, I killed the python interpreter process in the middle
- Latter calls to 'bzr pull' found a lock from the previous operation
- I broke the lock and started the pull again
- Since I have a connection that keeps breaking, I got the following stack trace
- My bzr is from the development head built using mingw32 on M$-XP

C:\users\dhruva\stub\repo\bzr\emacs\trunk
[dk]bzr break-lock
Break lock file:///C:/users/dhruva/stub/repo/bzr/emacs/trunk/.bzr/checkout/lock
held by <email address hidden> on host dhruva-lxp [process #3048]
locked 9 hours, 5 minutes ago? [y/n]: y
Break lock file:///C:/users/dhruva/stub/repo/bzr/emacs/trunk/.bzr/branch/lock
held by <email address hidden> on host dhruva-lxp [process #3048]
locked 9 hours, 5 minutes ago? [y/n]: y

C:\users\dhruva\stub\repo\bzr\emacs\trunk
[dk]bzr pull
Using saved location: http://bzr.notengoamigos.org/emacs/trunk/
bzr: ERROR: socket.error: (10054, 'Connection reset by peer')

Traceback (most recent call last):
  File "C:\python\Lib\site-packages\bzrlib\commands.py", line 834, in run_bzr_catch_errors
    return run_bzr(argv)
  File "C:\python\Lib\site-packages\bzrlib\commands.py", line 790, in run_bzr
    ret = run(*run_argv)
  File "C:\python\Lib\site-packages\bzrlib\commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "C:\python\Lib\site-packages\bzrlib\builtins.py", line 670, in run
    possible_transports=possible_transports)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\workingtree.py", line 1570, in pull
    possible_transports=possible_transports)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\branch.py", line 1714, in pull
    run_hooks=run_hooks)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\branch.py", line 1513, in pull
    self.update_revisions(source, stop_revision, overwrite=overwrite)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\branch.py", line 1465, in update_revisions
    self.fetch(other, stop_revision)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\branch.py", line 266, in fetch
    pb=nested_pb)
  File "C:\python\Lib\site-packages\bzrlib\repository.py", line 949, in fetch
    return inter.fetch(revision_id=revision_id, pb=pb, find_ghosts=find_ghosts)
  File "C:\python\Lib\site-packages\bzrlib\decorators.py", line 165, in write_locked
    return unbound(self, *args, **kwargs)
  File "C:\python\Lib\site-packages\bzrlib\repository.py", line 2759, in fetch
    revision_ids).pack()
  File "C:\python\Lib\site-packages\bzrlib\repofmt\pack_repo.py", line 589, in pack
    return self._create_pack_from_packs()
  File "C:\python\Lib\site-packages\bzrlib\repofmt\pack_repo.py", line 721, in _create_pack_from_packs
    self._copy_inventory_texts()
  File "C:\python\Lib\site-packages\bzrlib\repofmt\pack_repo.py", line 644, in _copy_inventory_texts
    total_items, readv_group_iter = self._least_readv_node_readv(inv_nodes)
  File "C:\python\Lib\site-packages\bzrlib\repofmt\pack_repo.py", line 870, in _least_readv_node_readv
    nodes = sorted(nodes)
  File "C:\python\Lib\site-packages\bzrlib\index.py", line 1074, in iter_entries
    for node in index.iter_entries(keys):
  File "C:\python\Lib\site-packages\bzrlib\index.py", line 479, in iter_entries
    self._lookup_keys_via_location, self._size, keys))
  File "C:\python\Lib\site-packages\bzrlib\bisect_multi.py", line 50, in bisect_multi_bytes
    search_results = content_lookup(search_keys)
  File "C:\python\Lib\site-packages\bzrlib\index.py", line 621, in _lookup_keys_via_location
    self._read_and_parse(readv_ranges)
  File "C:\python\Lib\site-packages\bzrlib\index.py", line 958, in _read_and_parse
    for offset, data in readv_data:
  File "C:\python\Lib\site-packages\bzrlib\transport\http\__init__.py", line 250, in _readv
    data = rfile.read(size)
  File "C:\python\Lib\site-packages\bzrlib\transport\http\response.py", line 209, in read
    data = self._file.read(limited)
  File "C:\python\lib\socket.py", line 309, in read
    data = self._sock.recv(recv_size)
  File "c:\python\lib\httplib.py", line 529, in read
    s = self.fp.read(amt)
  File "C:\python\lib\socket.py", line 309, in read
    data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')

bzr 1.4dev on python 2.5.2 (win32)
arguments: ['C:\\python\\Scripts\\bzr', 'pull']
encoding: 'cp1252', fsenc: 'mbcs', lang: None
plugins:
  launchpad C:\python\lib\site-packages\bzrlib\plugins\launchpad [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

[James Westby (james-w)]

On Fri, 2008-03-21 at 12:29 +0000, dhruva wrote:
> Public bug reported:
>
> - I had done a 'bzr pull' and since there is no information about progress and slow performance on emacs repo, I killed the python interpreter process in the middle
> - Latter calls to 'bzr pull' found a lock from the previous operation
> - I broke the lock and started the pull again
> - Since I have a connection that keeps breaking, I got the following stack trace
> - My bzr is from the development head built using mingw32 on M$-XP

Hi,

It's not exactly clear which point this bug report is about. Is it
that you get the traceback instead of a nicer message?

Thanks,

James

[hackworks (hackworks)]

On Fri, Mar 21, 2008 at 6:29 PM, James Westby <email address hidden> wrote:
> On Fri, 2008-03-21 at 12:29 +0000, dhruva wrote:
> > Public bug reported:
> >
> > - I had done a 'bzr pull' and since there is no information about progress and slow performance on emacs repo, I killed the python interpreter process in the middle
> > - Latter calls to 'bzr pull' found a lock from the previous operation
> > - I broke the lock and started the pull again
> > - Since I have a connection that keeps breaking, I got the following stack trace
> > - My bzr is from the development head built using mingw32 on M$-XP
>
> Hi,
>
> It's not exactly clear which point this bug report is about. Is it
> that you get the traceback instead of a nicer message?
>

If it is not serious, I would expect a nicer message. If it is
serious, I agree a stack trace is good. From my understanding, only
unhandled exceptions or unexpected errors throw stack trace. If every
error shows the stack trace, it becomes misleading about the severity
and like a "crying wolf story".

-dhruva

--
Contents reflect my personal views only!

[Vincent Ladeuil (vila)]

AFAICT, you're using the urllib http implementation (the relevant .bzr.log content may help to confirm this).

In that case, you're experimenting trouble with your connection to bzr.notengoamigos.org. The .bzr.log should shows that bzr has already retried the failing request and is failing a second time.

You can try 'bzr pull -Dhttp'.

.bzr.log will then contain more detailed info on the requests issued and the responses received.

'bzr version' will point you to exact location of your .bzr.log file.

[John A Meinel (jameinel)]

Well, it seems like the socket is raising

File "C:\python\lib\socket.py", line 309, in read
    data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')

Something like "socket.error(10054)". And we aren't recognizing the error code, so we just let it pass up the stack.

I think what we *should* do is just raise a ConnectionError at that point. That way the user will see a clean exception saying that the connection was broken.

Or even SocketConnectionError / ConnectionReset

All of them would still give a better error.SocketConnectionError gives decent host information, while ConnectionReset seems a perfect mapping for 10054.

[hackworks (hackworks)]

Hi,

On Sat, Mar 22, 2008 at 3:25 AM, John A Meinel <email address hidden> wrote:
> Well, it seems like the socket is raising
>
>
> File "C:\python\lib\socket.py", line 309, in read
> data = self._sock.recv(recv_size)
> error: (10054, 'Connection reset by peer')
>
> Something like "socket.error(10054)". And we aren't recognizing the
> error code, so we just let it pass up the stack.
>
> I think what we *should* do is just raise a ConnectionError at that
> point. That way the user will see a clean exception saying that the
> connection was broken.
>
> Or even SocketConnectionError / ConnectionReset
>
> All of them would still give a better error.SocketConnectionError gives
> decent host information, while ConnectionReset seems a perfect mapping
> for 10054.
>

I feel on such errors, bzr should sleep and retry a couple of times
before giving up. With WIFI based connectivity, the
probability of connection being intermittent is quite high. A retry
would be a nicer option.

-dhruva

--
Contents reflect my personal views only!

[Vincent Ladeuil (vila)]

_readv fails to catch the socket.errors (like the GET handling code does). Hence it did not retry.