Add a --s3-unencrypted-connection option to connect to S3 with regular HTTP (and not HTTPS)

Bug #433970 reported by sagi on 2009-09-21
This bug affects 3 people
Affects Status Importance Assigned to Milestone

Bug Description

Currently, when backing up to Amazon S3, an HTTPS connection is always used. This introduces a significant overhead and greatly slows down the upload.

It is possible to use regular HTTP connections by editing and passing is_secure=False to the S3Connection constructor, i.e.:
self.conn = S3Connection(is_secure=False)

By doing this I managed to backup performance that is several times faster.

It will be nice to be able to do this without editing the code, by passing a parameter to duplicity or setting an environment variable. I assume it won't make the backup less secure as the data is still encrypted.

Related branches

Martin Pool (mbp) wrote :

This would be very nice. It is indeed much faster, and will probably have a stronger effect the further you are from the Amazon datacentre. For me (in Australia) it cut the time to restore a particular backup from 40m to 6m.

I believe this would be a safe change: the s3 headers are digitally signed; the user data is either encrypted (or not) at the gpg level. The information exposure if there's a hostile connection somewhere between you and s3 is that they can see you're doing a duplicity backup and what the backup increments are. It seems to me this means the network is trusted as much as Amazon is trusted, which is a reasonable position.

Martin Pool (mbp) on 2010-10-13
Changed in duplicity:
assignee: nobody → Martin Pool (mbp)
importance: Undecided → Medium
status: New → In Progress
Changed in duplicity:
status: In Progress → Fix Committed
milestone: none → 0.6.11
assignee: Martin Pool (mbp) → nobody
Changed in duplicity:
status: Fix Committed → Fix Released
Martin Pool (mbp) on 2010-11-23
summary: - Add an option to connect to S3 with regular HTTP (and not HTTPS)
+ Add a --s3-unencrypted-connection option to connect to S3 with regular
+ HTTP (and not HTTPS)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers