Better support for new twisted.web.client.Agent in s3
Bug #767205 reported by
Drew Smathers
This bug affects 5 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
txAWS |
In Progress
|
Medium
|
Drew Smathers |
Bug Description
s3 currently uses getPage() for all operations. This can be detrimental to performance for doing things like uploading large files - the entire file has to be loaded into memory and written to the stream at once. getPage() also makes it harder (impossible) to do multi-part upload or chunked transfer encoding on a receiving end.
I would like to deprecate getPage() and write an adaptor layer to prefer the new Agent if available and downgrade to getPage() otherwise.
The initial implementation would provide simple String-based and File-based IBodyProducer implementations.
Related branches
lp:~djfroofy/txaws/newagent-767205
- Duncan McGreggor: Approve
- Drew Smathers: Needs Resubmitting
- Robert Collins: Abstain
-
Diff: 564 lines (+137/-52)3 files modifiedtxaws/client/base.py (+3/-1)
txaws/s3/client.py (+49/-25)
txaws/s3/tests/test_client.py (+85/-26)
Changed in txaws: | |
milestone: | 0.2 → 0.3 |
Changed in txaws: | |
assignee: | nobody → Drew Smathers (djfroofy) |
status: | New → In Progress |
To post a comment you must log in.
This sounds like a nice plan to me.