connection reset on backup to s3 target

[Daniel Waites]

Release: Ubuntu 14.04.2 LTS
Packages: deja-dup 30.0-0ubuntu4, duplicity 0.6.23-1ubuntu4.1
Gnome Conf Settings:

org.gnome.DejaDup last-restore ''
org.gnome.DejaDup periodic false
org.gnome.DejaDup full-backup-period 90
org.gnome.DejaDup backend 's3'
org.gnome.DejaDup last-run '2015-03-31T15:31:37.806498Z'
org.gnome.DejaDup nag-check '2015-03-31T15:31:37.807188Z'
org.gnome.DejaDup prompt-check 'disabled'
org.gnome.DejaDup root-prompt true
org.gnome.DejaDup include-list ['$HOME']
org.gnome.DejaDup exclude-list ['$TRASH', '$DOWNLOAD']
org.gnome.DejaDup last-backup '2015-03-31T15:31:37.806498Z'
org.gnome.DejaDup periodic-period 7
org.gnome.DejaDup delete-after 0
org.gnome.DejaDup.S3 id 'xxx'
org.gnome.DejaDup.S3 bucket 'xxx'
org.gnome.DejaDup.S3 folder 'daniel-ubuntu'
org.gnome.DejaDup.Rackspace username ''
org.gnome.DejaDup.Rackspace container 'daniel-ubuntu'
org.gnome.DejaDup.File path 'sftp:///home/daniel'
org.gnome.DejaDup.File short-name 'SYSTEM_DRV'
org.gnome.DejaDup.File uuid 'DC04859204857076'
org.gnome.DejaDup.File icon '. GThemedIcon drive-harddisk drive'
org.gnome.DejaDup.File relpath @ay []
org.gnome.DejaDup.File name 'ST1000DM003-1CH162: SYSTEM_DRV'
org.gnome.DejaDup.File type 'volume'

Backing up to s3 fails due to problems with python boto / duplicity which were fixed upstream but apparently have not made it to LTS. Backup to s3 fails due to connection reset error 104. I believe this is due to new signature requirements which were added to s3 requests approximately 1 year ago. A snippet of the error message from duplicity is as follows:

DUPLICITY: INFO 1
DUPLICITY: . Uploading s3+http://dwaitesdesktop.backups.mometrix/daniel-ubuntu/duplicity-full.20150331T154158Z.vol1.difftar.gpg to STANDARD Storage

DUPLICITY: WARNING 1
DUPLICITY: . Upload 's3+http://dwaitesdesktop.backups.mometrix/daniel-ubuntu/duplicity-full.20150331T154158Z.vol1.difftar.gpg' failed (attempt #5, reason: error: [Errno 104] Connection reset by peer)

DUPLICITY: DEBUG 1
DUPLICITY: . Backtrace of previous error: Traceback (innermost last):
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 221, in put
DUPLICITY: . num_cb=(max(2, 8 * globals.volsize / (1024 * 1024)))
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1306, in set_contents_from_filename
DUPLICITY: . encrypt_key=encrypt_key)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1237, in set_contents_from_file
DUPLICITY: . chunked_transfer=chunked_transfer, size=size)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 722, in send_file
DUPLICITY: . chunked_transfer=chunked_transfer, size=size)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in _send_file_internal
DUPLICITY: . query_args=query_args
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 547, in make_request
DUPLICITY: . retry_handler=retry_handler
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1017, in make_request
DUPLICITY: . retry_handler=retry_handler)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 894, in _mexe
DUPLICITY: . request.body, request.headers)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 812, in sender
DUPLICITY: . http_conn.send(chunk)
DUPLICITY: . File "/usr/lib/python2.7/httplib.py", line 805, in send
DUPLICITY: . self.sock.sendall(data)
DUPLICITY: . File "/usr/lib/python2.7/ssl.py", line 329, in sendall
DUPLICITY: . v = self.send(data[count:])
DUPLICITY: . File "/usr/lib/python2.7/ssl.py", line 298, in send
DUPLICITY: . v = self._sslobj.write(data)
DUPLICITY: . error: [Errno 104] Connection reset by peer
DUPLICITY: .

DUPLICITY: WARNING 1
DUPLICITY: . Giving up trying to upload s3+http://dwaitesdesktop.backups.mometrix/daniel-ubuntu/duplicity-full.20150331T154158Z.vol1.difftar.gpg after 5 attempts

DUPLICITY: DEBUG 1
DUPLICITY: . Releasing lockfile <lockfile.LinkFileLock instance at 0x7f42da498050>

DUPLICITY: DEBUG 1
DUPLICITY: . Removing still remembered temporary file /tmp/duplicity-lSqGjy-tempdir/mkstemp-1yAuZs-1

DUPLICITY: DEBUG 1
DUPLICITY: . Removing still remembered temporary file /tmp/duplicity-lSqGjy-tempdir/mktemp-gE3HFi-2

DUPLICITY: INFO 1
DUPLICITY: . Backend error detail: Traceback (most recent call last):
DUPLICITY: . File "/usr/bin/duplicity", line 1494, in <module>
DUPLICITY: . with_tempdir(main)
DUPLICITY: . File "/usr/bin/duplicity", line 1488, in with_tempdir
DUPLICITY: . fn()
DUPLICITY: . File "/usr/bin/duplicity", line 1337, in main
DUPLICITY: . do_backup(action)
DUPLICITY: . File "/usr/bin/duplicity", line 1458, in do_backup
DUPLICITY: . full_backup(col_stats)
DUPLICITY: . File "/usr/bin/duplicity", line 542, in full_backup
DUPLICITY: . globals.backend)
DUPLICITY: . File "/usr/bin/duplicity", line 424, in write_multivol
DUPLICITY: . (tdp, dest_filename, vol_num)))
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
DUPLICITY: . return self.__run_synchronously(fn, params)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
DUPLICITY: . ret = fn(*params)
DUPLICITY: . File "/usr/bin/duplicity", line 423, in <lambda>
DUPLICITY: . async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
DUPLICITY: . File "/usr/bin/duplicity", line 314, in put
DUPLICITY: . backend.put(tdp, dest_filename)
DUPLICITY: . File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 238, in put
DUPLICITY: . raise BackendException("Error uploading %s/%s" % (self.straight_url, remote_filename))
DUPLICITY: . BackendException: Error uploading s3+http://dwaitesdesktop.backups.mometrix/daniel-ubuntu/duplicity-full.20150331T154158Z.vol1.difftar.gpg
DUPLICITY: .

DUPLICITY: ERROR 23 BackendException
DUPLICITY: . BackendException: Error uploading s3+http://dwaitesdesktop.backups.mometrix/daniel-ubuntu/duplicity-full.20150331T154158Z.vol1.difftar.gpg

This is a widely reported issue with older versions of boto+duplicity.

[Federico (makexy)]

I have the same issue. S3 backup is broken.

deja-dup 34.0-1ubuntu3
Ubuntu Gnome 15.10
python-cloudfiles 1.7.11-3
python-boto 2.38.0-1ubuntu1
duplicity 0.7.02-1ubuntu1

[Michael Terry (mterry)]

This is not a deja-dup bug. Is this fixed these days on more recent systems?