Comment 11 for bug 948461

Some more data, after discussing with slangasek.
The attached script will run seemingly forever (100 times run) as it is, however if we remove the flag:
  -o=Acquire::http::Pipeline-Depth=0
to apt, then it will generally fail < 10 times.
(data collected on m1.large "right now", with mirrors set to http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/).

It would seem to me that the Pipeline-Depth is the source of the issue. See [1] for apt.conf description, and [2] for a quick google of 's3' 'http' 'pipeline' that gives some evidence that Pipeline and S3 aren't friends.

--
[1] http://manpages.ubuntu.com/manpages/precise/en/man5/apt.conf.5.html
[2] http://stackoverflow.com/questions/7752802/does-s3-support-http-pipeling