qsubprocess crashes with invalid bencode data from subprocess

[sebastian-s]

I wanted to branch git://github.com/chimeric/dokuwiki-template-arctic.git
The first time it went OK, the second I changed the destination directory and it failed with the message below. The new local repository/branch seems OK though.

bzr: ERROR: exceptions.ValueError: stream underflow

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/qbzr/lib/subprocess.py", line 498, in readStdout
    progress, transport_activity, messages = map(ensure_unicode, bencode.bdecode(line[14:]))
  File "_bencode_pyx.pyx", line 218, in _bencode_pyx.bdecode
  File "_bencode_pyx.pyx", line 83, in _bencode_pyx.Decoder.decode
  File "_bencode_pyx.pyx", line 105, in _bencode_pyx.Decoder._decode_object
  File "_bencode_pyx.pyx", line 186, in _bencode_pyx.Decoder._decode_list
  File "_bencode_pyx.pyx", line 102, in _bencode_pyx.Decoder._decode_object
  File "_bencode_pyx.pyx", line 164, in _bencode_pyx.Decoder._decode_string
ValueError: stream underflow

bzr 2.0.0 on python 2.6.4 (linux2)
arguments: ['/usr/bin/bzr', 'explorer']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.6/dist-packages/bzrlib/plugins/bzrtools [2.0.1]
  cvsps_import /home/seb/.bazaar/plugins/cvsps_import [unknown]
  dbus /usr/lib/python2.6/dist-packages/bzrlib/plugins/dbus [0.1.0dev]
  etckeeper /usr/lib/python2.6/dist-packages/bzrlib/plugins/etckeeper [unknown]
  explorer /usr/lib/python2.6/dist-packages/bzrlib/plugins/explorer [0.9.0]
  git /usr/lib/python2.6/dist-packages/bzrlib/plugins/git [0.4.1]
  gtk /usr/lib/python2.6/dist-packages/bzrlib/plugins/gtk [0.97.0.final]
  launchpad /usr/lib/python2.6/dist-packages/bzrlib/plugins/launchpad [2.0.0]
  netrc_credential_store /usr/lib/python2.6/dist-packages/bzrlib/plugins/netrc_credential_store [2.0.0]
  qbzr /usr/lib/python2.6/dist-packages/bzrlib/plugins/qbzr [0.14.4]
  svn /usr/lib/python2.6/dist-packages/bzrlib/plugins/svn [1.0.0dev]

[John A Meinel (jameinel)]

I don't know the specific cause, but looking at the traceback, this looks more like a bug in bzr-git that is provoking a bug in qbzr.

Specifically, I would guess that bzr-git somehow crashed, and that caused the output stream that was being watched to get truncated in the middle. This in turn caused qbzr to fail while parsing the stream.

I don't know what/how/why, but my initial guess would be that bzr-git segfaulted, which prevents anything buffered in stdout to get flushed.

Arguably qbzr should be having a try/except around the stream it is reading, so that it can give a nicer error to the user that the command was killed.

[sebastian-s (sebastian-s)]

Immediately after the error I tried again (no restart of explorer) and it worked. Just in case it helps.

[Alexander Belchenko (bialix)]

Agreed, QBzr should not traceback if received data from subprocess is broken.

[Alexander Belchenko (bialix)]

This error could be raised only if there is \n character send inside bencoded message. It could be only in the progress message from bzr-git. Jelmer, is it possible?

Another reason: the subprocess crashed behind the scene and send to front-end malformed string. In this case it may indicate very serious problem in underlying subprocess (segfault?).

So to fix this issue on QBzr side I'll just add try-except clause and show broken data as is in the status widget. So next time somebody will have similar problems we will have more information on what's going on.

[Jelmer Vernooij (jelmer)]

Yes, I wouldn't be surprised if there was a \n in a message somewhere.