"sslerror: The read operation timed out" with S3

Bug #505739 reported by Mike Rogers on 2010-01-11
This bug affects 3 people
Affects Status Importance Assigned to Milestone

Bug Description

I'm receiving the same error very often in 0.6.06 (and in all previously used versions). It sometimes requires re-running the backup process up to 5-10 times before it finally works.

Traceback (most recent call last):
 File "/usr/bin/duplicity", line 757, in ?
 File "/usr/bin/duplicity", line 750, in with_tempdir
 File "/usr/bin/duplicity", line 690, in main
 File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 538,
in set_values
   backend_filename_list = self.backend.list()
 File "/usr/lib/python2.4/site-packages/duplicity/backends/botobackend.py",
line 213, in list
   for k in self.bucket.list(prefix = self.key_prefix, delimiter = '/'):
 File "/usr/lib/python2.4/site-packages/boto/s3/bucketlistresultset.py", line
30, in bucket_lister
 File "/usr/lib/python2.4/site-packages/boto/s3/bucket.py", line 205, in
   body = response.read()
 File "/usr/lib/python2.4/httplib.py", line 460, in read
   return self._read_chunked(amt)
 File "/usr/lib/python2.4/httplib.py", line 509, in _read_chunked
   value += self._safe_read(chunk_left)
 File "/usr/lib/python2.4/httplib.py", line 555, in _safe_read
   chunk = self.fp.read(min(amt, MAXAMOUNT))
 File "/usr/lib/python2.4/httplib.py", line 971, in read
   s = self._read()
 File "/usr/lib/python2.4/httplib.py", line 947, in _read
   buf = self._ssl.read(self._bufsize)
sslerror: The read operation timed out

Gary Bernhardt (gary-bernhardt) wrote :

This was affecting me as well. Once I got to several hundred backups, it would fail very often. I don't understand why boto is timing out on S3 LISTs, but I sure know how to work around it. :)

The attached patch adds retry logic to the LIST requests. I just copied the retry code used in GET and PUT. I wanted to remove the duplication of retry code between all three, but the retry code isn't under test and I didn't want to risk breaking something.

Gary Bernhardt (gary-bernhardt) wrote :

With patch this time! :)

Mike Rogers (mikerogerz) wrote :

Thanks a lot... works perfectly. Let's hope they fix this for future versions.

Changed in duplicity:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Kenneth Loafman (kenneth-loafman)
milestone: none → 0.6.07
Changed in duplicity:
status: In Progress → Fix Committed
assignee: Kenneth Loafman (kenneth-loafman) → nobody
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers