When we do our monthly testing of backups stored on s3, duplicity crashes out on transitioned glacier files that are still listed from the bucket. We're working around it by not listing the files in a patch but if anyone has a better idea I'd be happy to implement it. I've linked our workaround.
Here's our system info:
Ubuntu 12.04.4 LTS
Python 2.7.3
duplicity 0.6.20
boto 2.4.1
Here's an example restore that crashes out:
Using archive dir: /mnt/cache2/e415c3e30cf174779e9311f3679408ac
Using backup name: e415c3e30cf174779e9311f3679408ac
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.u1backend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Main action: restore
================================================================================
duplicity 0.6.20 (October 28, 2012)
Args: /usr/local/bin/duplicity restore --timeout=300 --verbosity=8 --archive-dir=/mnt/cache2 --no-encryption s3+http://6087053278/uploads ./uploads-restored2 -v9
Linux madison-csm2os-Leader-bpBTXTxy 3.2.0-54-virtual #82-Ubuntu SMP Tue Sep 10 20:31:18 UTC 2013 x86_64 x86_64
/usr/bin/python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3]
================================================================================
Using temporary directory /tmp/duplicity-pSmd0h-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-pSmd0h-tempdir/mkstemp-Y2arEz-1
Temp has 7238836224 available, backup will use approx 34078720.
Listing s3+http://6087053278/uploads
Listed s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
... SNIP ...
Download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz failed (attempt #1, reason: S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>AAC655EFF3529702</RequestId><HostId>S3TPLPY2RmB9GeGNEKYVTxxS4R+tO+OrW+GIPRHqpzG3IsIDHEKj6ffpLCeT8HMB</HostId></Error>)
Backtrace of previous error: Traceback (innermost last):
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 241, in get
key.get_contents_to_filename(local_path.name)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1367, in get_contents_to_filename
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1314, in get_contents_to_file
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1190, in get_file
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 218, in open
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 170, in open_read
self.resp.reason, body)
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>AAC655EFF3529702</RequestId><HostId>S3TPLPY2RmB9GeGNEKYVTxxS4R+tO+OrW+GIPRHqpzG3IsIDHEKj6ffpLCeT8HMB</HostId></Error>
Downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
Download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz failed (attempt #2, reason: AttributeError: 'NoneType' object has no attribute 'has_key')
Backtrace of previous error: Traceback (innermost last):
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 241, in get
key.get_contents_to_filename(local_path.name)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1367, in get_contents_to_filename
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1314, in get_contents_to_file
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1226, in get_file
if self.size is None and not torrent and not headers.has_key("Range"):
AttributeError: 'NoneType' object has no attribute 'has_key'
Downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
Download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz failed (attempt #3, reason: S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>05BFAA5798376DAE</RequestId><HostId>bi8dKmIPHCg1yqdflcNlb/R/gGDNkZO9dCtM9j4h/CJQ5XADn4k9c+Ew7xUCbXD1</HostId></Error>)
Backtrace of previous error: Traceback (innermost last):
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 241, in get
key.get_contents_to_filename(local_path.name)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1367, in get_contents_to_filename
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1314, in get_contents_to_file
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1190, in get_file
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 218, in open
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 170, in open_read
self.resp.reason, body)
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>05BFAA5798376DAE</RequestId><HostId>bi8dKmIPHCg1yqdflcNlb/R/gGDNkZO9dCtM9j4h/CJQ5XADn4k9c+Ew7xUCbXD1</HostId></Error>
Downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
Download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz failed (attempt #4, reason: AttributeError: 'NoneType' object has no attribute 'has_key')
Backtrace of previous error: Traceback (innermost last):
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 241, in get
key.get_contents_to_filename(local_path.name)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1367, in get_contents_to_filename
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1314, in get_contents_to_file
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1226, in get_file
if self.size is None and not torrent and not headers.has_key("Range"):
AttributeError: 'NoneType' object has no attribute 'has_key'
Downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
Download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz failed (attempt #5, reason: S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>8FDBAFB522AD2FF8</RequestId><HostId>MdYu9030Wmww1SBIbASAIBY9KzckLIwCd6ZxZpAxHsh4lehSLKVhifF28F9vZw8c</HostId></Error>)
Backtrace of previous error: Traceback (innermost last):
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 241, in get
key.get_contents_to_filename(local_path.name)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1367, in get_contents_to_filename
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1314, in get_contents_to_file
response_headers=response_headers)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1190, in get_file
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 218, in open
override_num_retries=override_num_retries)
File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 170, in open_read
self.resp.reason, body)
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>8FDBAFB522AD2FF8</RequestId><HostId>MdYu9030Wmww1SBIbASAIBY9KzckLIwCd6ZxZpAxHsh4lehSLKVhifF28F9vZw8c</HostId></Error>
Giving up trying to download s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz after 5 attempts
Removing still remembered temporary file /tmp/duplicity-pSmd0h-tempdir/mktemp-7lBmhI-2
Removing still remembered temporary file /tmp/duplicity-pSmd0h-tempdir/mkstemp-Y2arEz-1
Backend error detail: Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 1403, in <module>
with_tempdir(main)
File "/usr/local/bin/duplicity", line 1396, in with_tempdir
fn()
File "/usr/local/bin/duplicity", line 1272, in main
sync_archive(decrypt)
File "/usr/local/bin/duplicity", line 1072, in sync_archive
copy_to_local(fn)
File "/usr/local/bin/duplicity", line 1013, in copy_to_local
fileobj = globals.backend.get_fileobj_read(fn)
File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 555, in get_fileobj_read
self.get(filename, tdp)
File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 256, in get
raise BackendException("Error downloading %s/%s" % (self.straight_url, remote_filename))
BackendException: Error downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
BackendException: Error downloading s3+http://6087053278/uploads/duplicity-full-signatures.20121108T103500Z.sigtar.gz
