bzr fails with smart protocol error
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Bazaar |
Fix Released
|
Medium
|
Unassigned |
Bug Description
When trying to branch from a remote repo using the following;
$ bzr version
Bazaar (bzr) 1.6.1
Python interpreter: /home/username/
Python standard library: /home/username/
bzrlib: /home/username/
Bazaar configuration: /home/username/
Bazaar log file: /home/username/
$ bzr -Dhpss branch bzr+ssh:
I get the following error,
0.068 encoding stdout as sys.stdout encoding 'UTF-8'
0.069 bzr arguments: [u'-Dhpss', u'branch', u'bzr+ssh:
0.070 looking for plugins in /home/username/
0.070 looking for plugins in /home/username/
0.070 Plugin name __init__ already loaded
0.070 Plugin name __init__ already loaded
0.103 WARNING: using slower ElementTree; consider installing cElementTree and make sure it's on your PYTHONPATH
0.147 looking for plugins in /home/username/
0.162 encoding stdout as sys.stdout encoding 'UTF-8'
0.187 hpss: Built a new medium: SmartSSHClientM
0.194 hpss call: 'BzrDir.open', 'home/username/
0.194 (to bzr+ssh:
0.234 ssh implementation is OpenSSH
8.857 result: ('yes',)
8.858 hpss call: 'BzrDir.
8.858 (to bzr+ssh:
8.879 result: ('ok', '')
8.879 hpss call: 'BzrDir.
8.879 (to bzr+ssh:
9.270 result: ('ok', '..', 'no', 'no', 'no')
9.272 hpss call: 'BzrDir.open', 'home/username/
9.272 (to bzr+ssh:
9.291 result: ('yes',)
9.292 hpss call: 'BzrDir.
9.292 (to bzr+ssh:
9.312 result: ('ok', '', 'no', 'no', 'no')
9.312 hpss call: 'Repository.
9.312 (to bzr+ssh:
9.461 result: ('yes',)
9.462 hpss call: 'Branch.
9.462 (to bzr+ssh:
9.781 result: ('ok', '95', '<email address hidden>')
9.784 hpss call: 'get', '/home/
9.784 (to bzr+ssh:
9.848 result: ('ok',)
9.848 35 body bytes read
9.849 hpss call: 'get', '/home/
9.849 (to bzr+ssh:
9.868 result: ('ok',)
9.868 34 body bytes read
9.869 hpss call: 'stat', '/home/
9.869 (to bzr+ssh:
9.888 result: ('stat', '4096', '040755')
9.889 hpss call: 'get', '/home/
9.889 (to bzr+ssh:
9.908 result: ('NoSuchFile', '/home/
9.909 hpss call: 'BzrDir.open', 'home/username/
9.909 (to bzr+ssh:
9.929 result: ('yes',)
9.930 hpss call: 'BzrDir.
9.930 (to bzr+ssh:
9.949 result: ('ok', '', 'no', 'no', 'no')
9.949 hpss call: 'Repository.
9.949 (to bzr+ssh:
9.969 result: ('yes',)
9.981 hpss call: 'get', '/home/
9.981 (to bzr+ssh:
10.026 result: ('ok',)
10.026 40 body bytes read
10.032 hpss call: 'BzrDir.
10.032 (to bzr+ssh:
10.053 result: ('ok', '')
10.053 hpss call: 'BzrDir.
10.054 (to bzr+ssh:
10.074 result: ('ok', '..', 'no', 'no', 'no')
10.075 hpss call: 'BzrDir.open', 'home/username/
10.075 (to bzr+ssh:
10.094 result: ('yes',)
10.095 hpss call: 'BzrDir.
10.095 (to bzr+ssh:
10.115 result: ('ok', '', 'no', 'no', 'no')
10.115 hpss call: 'Repository.
10.115 (to bzr+ssh:
10.135 result: ('yes',)
10.139 hpss call: 'get', '/home/
10.140 (to bzr+ssh:
10.159 result: ('ok',)
10.159 35 body bytes read
10.160 hpss call: 'get', '/home/
10.160 (to bzr+ssh:
10.180 result: ('ok',)
10.180 49 body bytes read
10.180 hpss call: 'stat', '/home/
10.180 (to bzr+ssh:
10.200 result: ('stat', '4096', '040755')
10.201 hpss call: 'get', '/home/
10.202 (to bzr+ssh:
10.222 result: ('ok',)
10.222 415 body bytes read
10.224 Using fetch logic to copy between KnitPackReposit
10.225 hpss call w/readv: 'readv', '/home/
10.225 5 bytes in readv request
10.275 result: ('readv',)
10.275 276 body bytes read
10.275 hpss call: 'get', '/home/
10.275 (to bzr+ssh:
10.295 result: ('ok',)
10.295 276 body bytes read
10.296 hpss call w/readv: 'readv', '/home/
10.296 5 bytes in readv request
10.346 result: ('readv',)
10.346 273 body bytes read
10.346 hpss call: 'get', '/home/
10.346 (to bzr+ssh:
10.366 result: ('ok',)
10.366 273 body bytes read
10.366 hpss call w/readv: 'readv', '/home/
10.366 5 bytes in readv request
10.466 result: ('readv',)
10.466 279 body bytes read
10.466 hpss call: 'get', '/home/
10.466 (to bzr+ssh:
10.487 result: ('ok',)
10.488 279 body bytes read
10.488 hpss call w/readv: 'readv', '/home/
10.488 5 bytes in readv request
10.528 result: ('readv',)
10.528 201 body bytes read
10.528 hpss call: 'get', '/home/
10.528 (to bzr+ssh:
10.547 result: ('ok',)
10.548 201 body bytes read
10.548 hpss call w/readv: 'readv', '/home/
10.548 5 bytes in readv request
10.587 result: ('readv',)
10.588 192 body bytes read
10.588 hpss call: 'get', '/home/
10.588 (to bzr+ssh:
10.607 result: ('ok',)
10.607 192 body bytes read
10.608 hpss call w/readv: 'readv', '/home/
10.608 7 bytes in readv request
10.659 result: ('readv',)
10.677 10247 body bytes read
10.680 hpss call w/readv: 'readv', '/home/
10.681 5 bytes in readv request
10.712 result: ('readv',)
10.713 265 body bytes read
10.713 hpss call: 'get', '/home/
10.713 (to bzr+ssh:
10.732 result: ('ok',)
10.732 265 body bytes read
10.751 hpss call: 'get', '/home/
10.751 (to bzr+ssh:
10.790 result: ('ok',)
10.791 10247 body bytes read
10.796 hpss call w/readv: 'readv', '/home/
10.796 51 bytes in readv request
10.859 result: ('readv',)
10.916 50205 body bytes read
10.945 hpss call w/readv: 'readv', '/home/
10.945 5 bytes in readv request
10.980 result: ('readv',)
10.980 461 body bytes read
10.980 hpss call w/readv: 'readv', '/home/
10.981 12 bytes in readv request
11.006 result: ('readv',)
11.006 485 body bytes read
11.007 hpss call w/readv: 'readv', '/home/
11.007 5 bytes in readv request
11.036 result: ('readv',)
11.037 485 body bytes read
11.037 hpss call w/readv: 'readv', '/home/
11.037 5 bytes in readv request
11.057 result: ('readv',)
11.057 480 body bytes read
11.057 hpss call w/readv: 'readv', '/home/
11.058 12 bytes in readv request
11.079 result: ('readv',)
11.079 465 body bytes read
11.079 hpss call w/readv: 'readv', '/home/
11.080 5 bytes in readv request
11.111 result: ('readv',)
11.112 462 body bytes read
11.112 hpss call w/readv: 'readv', '/home/
11.112 5 bytes in readv request
11.133 result: ('readv',)
11.133 206 body bytes read
11.134 hpss call: 'get', '/home/
11.134 (to bzr+ssh:
11.154 result: ('ok',)
11.154 206 body bytes read
11.154 hpss call w/readv: 'readv', '/home/
11.154 7 bytes in readv request
11.184 result: ('readv',)
11.186 10971 body bytes read
11.189 hpss call: 'get', '/home/
11.189 (to bzr+ssh:
11.211 result: ('ok',)
11.211 10971 body bytes read
11.219 hpss call w/readv: 'readv', '/home/
11.219 5 bytes in readv request
11.239 result: ('readv',)
11.239 284 body bytes read
11.239 hpss call: 'get', '/home/
11.239 (to bzr+ssh:
11.259 result: ('ok',)
11.259 284 body bytes read
11.259 hpss call w/readv: 'readv', '/home/
11.259 5 bytes in readv request
11.280 result: ('readv',)
11.280 268 body bytes read
11.280 hpss call: 'get', '/home/
11.281 (to bzr+ssh:
11.301 result: ('ok',)
11.301 268 body bytes read
11.302 hpss call w/readv: 'readv', '/home/
11.302 5 bytes in readv request
11.322 result: ('readv',)
11.322 278 body bytes read
11.323 hpss call: 'get', '/home/
11.323 (to bzr+ssh:
11.342 result: ('ok',)
11.342 278 body bytes read
11.342 hpss call w/readv: 'readv', '/home/
11.343 5 bytes in readv request
11.363 result: ('readv',)
11.363 197 body bytes read
11.363 hpss call: 'get', '/home/
11.363 (to bzr+ssh:
11.383 result: ('ok',)
11.383 197 body bytes read
11.384 hpss call w/readv: 'readv', '/home/
11.384 5 bytes in readv request
11.416 result: ('readv',)
11.417 279 body bytes read
11.417 hpss call: 'get', '/home/
11.417 (to bzr+ssh:
11.436 result: ('ok',)
11.437 279 body bytes read
11.439 hpss call w/readv: 'readv', '/home/
11.439 12 bytes in readv request
11.459 result: ('readv',)
11.459 327 body bytes read
11.460 hpss call w/readv: 'readv', '/home/
11.461 55 bytes in readv request
11.622 result: ('readv',)
12.133 1187736 body bytes read
12.859 hpss call w/readv: 'readv', '/home/
12.859 12 bytes in readv request
12.879 result: ('readv',)
12.879 326 body bytes read
12.882 hpss call w/readv: 'readv', '/home/
12.882 5 bytes in readv request
12.901 result: ('readv',)
12.902 362 body bytes read
12.902 hpss call w/readv: 'readv', '/home/
12.902 12 bytes in readv request
12.922 result: ('readv',)
12.922 343 body bytes read
12.923 hpss call w/readv: 'readv', '/home/
12.923 12 bytes in readv request
12.942 result: ('readv',)
12.942 364 body bytes read
12.942 hpss call w/readv: 'readv', '/home/
12.942 5 bytes in readv request
12.962 result: ('readv',)
12.962 358 body bytes read
13.173 hpss call w/readv: 'readv', '/home/
13.173 4 bytes in readv request
13.194 result: ('readv',)
13.195 60 body bytes read
13.195 hpss call: 'get', '/home/
13.195 (to bzr+ssh:
13.214 result: ('ok',)
13.215 60 body bytes read
13.225 hpss call w/readv: 'readv', '/home/
13.226 7 bytes in readv request
13.281 result: ('readv',)
13.373 65536 body bytes read
13.395 hpss call: 'get', '/home/
13.395 (to bzr+ssh:
13.502 result: ('ok',)
14.470 2803849 body bytes read
15.736 hpss call w/readv: 'readv', '/home/
15.736 5 bytes in readv request
15.756 result: ('readv',)
15.756 292 body bytes read
15.757 hpss call: 'get', '/home/
15.757 (to bzr+ssh:
15.776 result: ('ok',)
15.777 292 body bytes read
15.777 hpss call w/readv: 'readv', '/home/
15.777 5 bytes in readv request
15.798 result: ('readv',)
15.798 293 body bytes read
15.798 hpss call: 'get', '/home/
15.798 (to bzr+ssh:
15.817 result: ('ok',)
15.817 293 body bytes read
15.818 hpss call w/readv: 'readv', '/home/
15.818 4 bytes in readv request
15.839 result: ('readv',)
15.839 60 body bytes read
15.839 hpss call: 'get', '/home/
15.839 (to bzr+ssh:
15.859 result: ('ok',)
15.859 60 body bytes read
15.860 hpss call w/readv: 'readv', '/home/
15.860 4 bytes in readv request
15.880 result: ('readv',)
15.880 60 body bytes read
15.880 hpss call: 'get', '/home/
15.880 (to bzr+ssh:
15.899 result: ('ok',)
15.899 60 body bytes read
15.900 hpss call w/readv: 'readv', '/home/
15.900 5 bytes in readv request
15.921 result: ('readv',)
15.921 172 body bytes read
15.921 hpss call: 'get', '/home/
15.921 (to bzr+ssh:
15.952 result: ('ok',)
15.952 172 body bytes read
16.772 hpss call w/readv: 'readv', '/home/
16.773 6681 bytes in readv request
40.295 result: ('error', '')
40.306 Traceback (most recent call last):
File "/home/
return run_bzr(argv)
File "/home/
ret = run(*run_argv)
File "/home/
return self.run(
File "/home/
hardlink=
File "/home/
result_
File "/home/
find_
File "/home/
find_
File "/home/
result = unbound(self, *args, **kwargs)
File "/home/
revision_
File "/home/
return self._create_
File "/home/
self.
File "/home/
self.
File "/home/
write_index, output_lines, pb, readv_group_iter, total_items):
File "/home/
reader = pack.make_
File "/home/
result = ContainerReader
File "/home/
return self._readv(
File "/home/
self.
File "/home/
raise errors.
SmartProtocolError: Generic bzr smart protocol error:
40.306 return code 3
The server is replying to a readv request with ('error', ''). There's not much the client can do with that except report the useless error to the user.
So the question is why does the server send such a useless error. The .bzr.log file on the server (in that user's home directory) should contain the exception. Please attach that .bzr.log (or the relevant parts of it) to this bug, Also, which version of bzr is on the server?
I think I can guess what's going on, but I need the log file to confirm. Judging from this part of the client's log:
16.773 6681 bytes in readv request
40.295 result: ('error', '')
I'm guessing it's a MemoryError. A 6681 byte readv request is quite large, and it it apparently took nearly 25 seconds between sending the request and receiving the failure. Looking at the error handling code, it appears bzr serve will serialise a MemoryError like that.
Also, there's a bug in how the bzr smart server buffers readv replies in memory before sending them to the client. If you upgrade bzr on the client then this bug will probably be fixed, because the current version of the bzr client will make smaller readv requests to avoid this bug.