Better support for parameterizing standard HTTP headers in s3 apis

[Drew Smathers]

This is more of a placeholder (and may be a duplicate of another ticket), but currently there is no way to supply standard HTTP headers applicable to S3 in txaws.s3.client.

These could be tallied for each relevant command and supplied as keyword arguments, for example.

As individual keyword args:

    client.put_object(..., cache_control=x, expires=y, ...)

Or just headers object or dict which could be validated for each method:

    client.put_object(..., headers=x)

[Duncan McGreggor (oubiwann)]

Awesome. This is something I'm going to need very soon, as I'm adding support for canned acls.

Also, note that I'm traveling this weekend and might not get to any reviews until I'm back home.

d

Sent from my iPhone

On Feb 10, 2012, at 12:43 PM, Drew Smathers <email address hidden> wrote:

> Public bug reported:
>
> This is more of a placeholder (and may be a duplicate of another
> ticket), but currently there is no way to supply standard HTTP headers
> applicable to S3 in txaws.s3.client.
>
> These could be tallied for each relevant command and supplied as keyword
> arguments, for example.
>
> As individual keyword args:
>
> client.put_object(..., cache_control=x, expires=y, ...)
>
> Or just headers object or dict which could be validated for each method:
>
> client.put_object(..., headers=x)
>
> ** Affects: txaws
> Importance: Undecided
> Status: New
>
> ** Description changed:
>
> This is more of a placeholder (and may be a duplicate of another
> ticket), but currently there is no way to supply standard HTTP headers
> applicable to S3 in txaws.s3.client.
>
> These could be tallied for each relevant command and supplied as keyword
> arguments, for example.
>
> - For example:
> + As individual keyword args:
>
> - client.put_object(..., cache_control=x, expires=y, ...)
> + client.put_object(..., cache_control=x, expires=y, ...)
>
> - Or pass a headers dict or object:
> + Or just headers object or dict which could be validated for each method:
>
> - client.put_object(..., headers=x)
> + client.put_object(..., headers=x)
>
> --
> You received this bug notification because you are subscribed to txAWS.
> https://bugs.launchpad.net/bugs/930359
>
> Title:
> Better support for parameterizing standard HTTP headers in s3 apis
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/txaws/+bug/930359/+subscriptions

[Drew Smathers (djfroofy)]

Ok, no urgency on reviewing the ticket but I will just say this current ticket is lower in priority for my company (we really need new Agent support for streaming + multipart features). I'll get back to this ticket after those.

I would like to suggest we change the API to take "headers" argument instead of piecemeal. If you have other thoughts, let me know. For backwards compat we could deprecate usage of "content_type" keyword args etc and do deprecation warnings.

If we do decide to use headers keyword arg, my next questions are:

1. Should we require Headers object from twisted?
2. Plain dict?
3. Either and do type checking to coerce to Headers if dict?

Note that eventually we have to coerce to the Headers object to use with Agent API so far as I know.

[Drew Smathers (djfroofy)]

s/ticket/tickets/ ^^^

I have a few queued up for review for Agent support, s3 apis for plugging in body_producer or receiver, mutlipart commands, ...

[Duncan McGreggor (oubiwann)]

On Sat, Feb 11, 2012 at 11:52 AM, Drew Smathers
<email address hidden> wrote:
>
> I would like to suggest we change the API to take "headers" argument
> instead of piecemeal.

My thinking as well.

> If you have other thoughts, let me know. For
> backwards compat we could deprecate usage of "content_type" keyword args
> etc and do deprecation warnings.
>
> If we do decide to use headers keyword arg, my next questions are:
>
> 1. Should we require Headers object from twisted?
> 2. Plain dict?

That's what my gut tells me...

> 3. Either and do type checking to coerce to Headers if dict?

Yeah, that might be a ways to go...

Currently the headers are set in a couple locations:

1) txaws.ec2.client.Query.submit (built in local scope, ultimately
sent to getPage in kwargs)
2) txaws.s3.client.S3Client (amz_headers in various S3Client methods
and Query methods)

It would be nice if we could define a sane and unified mechanism for
setting headers across the entire library, generalizing in
txaws.client.base and extending/modifying as necessary in subclasses.

> Note that eventually we have to coerce to the Headers object to use with
> Agent API so far as I know.

/me nods

Good to know -- thanks!

[Drew Smathers (djfroofy)]

Looking at the code, I'm finding it hard to envision a sane and unified way of setting headers. I've done 0 development with ec2 development so I may be missing some subtle commonalities between the two. Just looking at the code though, the header setting in ec2.client and s3.client seem radically different. However, this is something I'd like to discuss more if after focussing first on s3.