osutils.send_all() loops endlessly sending 0 bytes if using paramiko for ssh
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Bazaar |
Fix Released
|
High
|
John A Meinel | ||
2.5 |
Fix Released
|
High
|
John A Meinel |
Bug Description
I'm investigating the ConnectionReset issues we've been seeing.
It appears that if you set BZR_SSH=paramiko and then run this:
python -c "import time; from bzrlib import breakin, branch, plugin, initialize; e = initialize(); e.__enter__(); plugin.
b = branch.
print b.last_revision(); b.unlock();
for i in range(35):
print i
time.sleep(10)
b.lock_read()
print b.last_revision()
b.unlock()
"
Then the code loops endlessly in this loop:
def send_all(sock, bytes, report_
...
sent_total = 0
byte_count = len(bytes)
while sent_total < byte_count:
try:
sent = sock.send(
except socket.error, e:
if e.args[0] != errno.EINTR:
else:
In particular sock.send() in this case returns '0' bytes sent. But it doesn't actually give an error. I think we should probably put a trap on allowing 0 bytes to be sent for more than X tries. I don't know if it is ever genuine for sock.send to return 0 bytes, but we shouldn't try to send forever and always send 0 bytes.
Related branches
- Richard Wilbur: Needs Fixing
- Martin Packman (community): Approve
-
Diff: 68 lines (+34/-1)3 files modifiedbzrlib/osutils.py (+5/-1)
bzrlib/tests/test_osutils.py (+22/-0)
doc/en/release-notes/bzr-2.5.txt (+7/-0)
summary: |
- ssh paramiko loops endlessly sending 0 bytes + osutils.send_all() loops endlessly sending 0 bytes if using paramiko for + ssh |
Changed in bzr: | |
assignee: | nobody → John A Meinel (jameinel) |
Changed in bzr: | |
status: | Confirmed → Fix Released |
milestone: | none → 2.5.2 |
milestone: | 2.5.2 → 2.6b3 |
Changed in bzr: | |
milestone: | 2.6b3 → 2.6.0 |
A first draft fix would say "if we have sent 0 bytes after X tries, fail". Looking at the man page, send() should block until the bytes requested are sent, or give other sorts of errors about the message size being too big, etc.
I would probably do something like:
num_zero_sends = 0
max_zero_sends = 3 # config entry?
while sent_bytes < ...: ConnectionReset (...)
if sent == 0:
num_zero_sends += 1
if num_zero_sends > max_zero_sends:
raise errors.
else:
num_zero_sends = 0
That allows us to have a transient '0' bytes sent at any point, but in the end
we must get forward progress or we will fail properly.