"sslerror: The read operation timed out" with S3

Bug #505739 reported by Mike Rogers on 2010-01-11
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Duplicity
Medium
Unassigned

Bug Description

I'm receiving the same error very often in 0.6.06 (and in all previously used versions). It sometimes requires re-running the backup process up to 5-10 times before it finally works.

Traceback (most recent call last):
 File "/usr/bin/duplicity", line 757, in ?
   with_tempdir(main)
 File "/usr/bin/duplicity", line 750, in with_tempdir
   fn()
 File "/usr/bin/duplicity", line 690, in main
   globals.archive_dir).set_values()
 File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 538,
in set_values
   backend_filename_list = self.backend.list()
 File "/usr/lib/python2.4/site-packages/duplicity/backends/botobackend.py",
line 213, in list
   for k in self.bucket.list(prefix = self.key_prefix, delimiter = '/'):
 File "/usr/lib/python2.4/site-packages/boto/s3/bucketlistresultset.py", line
30, in bucket_lister
   delimiter=delimiter)
 File "/usr/lib/python2.4/site-packages/boto/s3/bucket.py", line 205, in
get_all_keys
   body = response.read()
 File "/usr/lib/python2.4/httplib.py", line 460, in read
   return self._read_chunked(amt)
 File "/usr/lib/python2.4/httplib.py", line 509, in _read_chunked
   value += self._safe_read(chunk_left)
 File "/usr/lib/python2.4/httplib.py", line 555, in _safe_read
   chunk = self.fp.read(min(amt, MAXAMOUNT))
 File "/usr/lib/python2.4/httplib.py", line 971, in read
   s = self._read()
 File "/usr/lib/python2.4/httplib.py", line 947, in _read
   buf = self._ssl.read(self._bufsize)
sslerror: The read operation timed out

Gary Bernhardt (gary-bernhardt) wrote :

This was affecting me as well. Once I got to several hundred backups, it would fail very often. I don't understand why boto is timing out on S3 LISTs, but I sure know how to work around it. :)

The attached patch adds retry logic to the LIST requests. I just copied the retry code used in GET and PUT. I wanted to remove the duplication of retry code between all three, but the retry code isn't under test and I didn't want to risk breaking something.

Gary Bernhardt (gary-bernhardt) wrote :

With patch this time! :)

Mike Rogers (mikerogerz) wrote :

Thanks a lot... works perfectly. Let's hope they fix this for future versions.

Changed in duplicity:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Kenneth Loafman (kenneth-loafman)
milestone: none → 0.6.07
Changed in duplicity:
status: In Progress → Fix Committed
assignee: Kenneth Loafman (kenneth-loafman) → nobody
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers