Large Objects/Manifests ignore allowed_headers (Content-Encoding, Content-Disposition, ...)
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Object Storage (swift) |
Fix Released
|
High
|
John Dickinson | ||
Bug Description
This was originally Rackspace Cloud Ticket 167610.
It looks like custom headers like Content-Encoding and Content-Disposition and the CORS header are ignored for Large Objects which are glued together with a X-Object-Manifest.
You can find a testcase in my manifest-testcase branch of my fork of php-cloudfiles at https:/
It will create two objects in the container "test": manifest-test/data and manifest-test/link. The latter has a X-Object-Manifest which points to the first. Additionally, it will set the custom header Content-Disposition (and optionally Content-Encoding, see below). It will also set the container to public and print the public URI for the link in the end.
If you request that URI, you will notice that the Content-Disposition header is not included. If you request the data file instead, it is.
It gets worse if the Large Object is stored gzip compressed (enable the gzip option in the ini file to test this) since the Content-Encoding will not be sent to the user agent.
A quick glance at the Code shows that the swift.proxy.server copies the X-Object-Meta-* headers but not the stuff stored in allowed_headers:
https:/
It probably needs the functionality implemented in swift.obj.server:
https:/
Here is a sample test run:
mss@TreeHouse:
++ dirname ./manifest-test.sh
+ cd .
++ basename ./manifest-test.sh .sh
+ exec php manifest-test.php
* About to connect() to storage101.
* Trying 94.236.56.96... * connected
* Connected to storage101.
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using RC4-SHA
* Server certificate:
* subject: O=storage101.
* start date: 2010-09-24 00:00:00 GMT
* expire date: 2012-09-23 23:59:59 GMT
* common name: storage101.
* issuer: C=US; O=Thawte, Inc.; OU=Domain Validated SSL; CN=Thawte DV SSL CA
* SSL certificate verify ok.
> PUT /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 202 Accepted
< Content-Length: 58
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx2b94482108754
< Date: Fri, 19 Aug 2011 15:39:25 GMT
<
* Connection #0 to host storage101.
* About to connect() to cdn3.clouddrive.com port 443 (#0)
* Trying 94.236.56.97... * connected
* Connected to cdn3.clouddrive.com (94.236.56.97) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: O=cdn3.
* start date: 2010-10-06 00:00:00 GMT
* expire date: 2012-10-05 23:59:59 GMT
* common name: cdn3.clouddrive.com (matched)
* issuer: C=US; O=Thawte, Inc.; OU=Domain Validated SSL; CN=Thawte DV SSL CA
* SSL certificate verify ok.
> HEAD /v1/MossoCloudF
Host: cdn3.clouddrive.com
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 204 No Content
< Date: Fri, 19 Aug 2011 15:39:25 GMT
< Server: Apache
< X-CDN-URI: http://
< X-CDN-SSL-URI: https:/
< X-CDN-Enabled: True
< X-TTL: 86400
< X-Log-Retention: True
< Connection: close
< Content-Type: text/plain; charset=UTF-8
<
* Closing connection #0
* About to connect() to cdn3.clouddrive.com port 443 (#0)
* Trying 94.236.56.97... * connected
* Connected to cdn3.clouddrive.com (94.236.56.97) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: O=cdn3.
* start date: 2010-10-06 00:00:00 GMT
* expire date: 2012-10-05 23:59:59 GMT
* common name: cdn3.clouddrive.com (matched)
* issuer: C=US; O=Thawte, Inc.; OU=Domain Validated SSL; CN=Thawte DV SSL CA
* SSL certificate verify ok.
> POST /v1/MossoCloudF
Host: cdn3.clouddrive.com
Accept: */*
X-CDN-Enabled: True
X-TTL: 86400
X-Log-Retention: True
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 202 Accepted
< Date: Fri, 19 Aug 2011 15:39:26 GMT
< Server: Apache
< X-CDN-URI: http://
< X-CDN-SSL-URI: https:/
< Content-Length: 0
< Connection: close
< Content-Type: text/plain; charset=UTF-8
<
* Closing connection #0
* About to connect() to storage101.
* Trying 94.236.56.96... * connected
* Connected to storage101.
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using RC4-SHA
* Server certificate:
* subject: O=storage101.
* start date: 2010-09-24 00:00:00 GMT
* expire date: 2012-09-23 23:59:59 GMT
* common name: storage101.
* issuer: C=US; O=Thawte, Inc.; OU=Domain Validated SSL; CN=Thawte DV SSL CA
* SSL certificate verify ok.
> HEAD /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 404 Not Found
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx2d2b1e1cf3324
< Date: Fri, 19 Aug 2011 15:39:26 GMT
<
* Connection #0 to host storage101.
* About to connect() to storage101.
* Trying 94.236.56.96... * connected
* Connected to storage101.
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using RC4-SHA
* Server certificate:
* subject: O=storage101.
* start date: 2010-09-24 00:00:00 GMT
* expire date: 2012-09-23 23:59:59 GMT
* common name: storage101.
* issuer: C=US; O=Thawte, Inc.; OU=Domain Validated SSL; CN=Thawte DV SSL CA
* SSL certificate verify ok.
> PUT /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Object-Meta-Data: test
Content-
Content-Encoding: gzip
ETag: 38962e22d2a9e97
Content-Type: application/x-gzip
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
Content-Length: 24
Expect: 100-continue
< HTTP/1.1 100 Continue
< HTTP/1.1 201 Created
< Content-Length: 118
< Content-Type: text/html; charset=UTF-8
< Etag: 38962e22d2a9e97
< Last-Modified: Fri, 19 Aug 2011 15:39:26 GMT
< X-Trans-Id: txeb60af8276ab4
< Date: Fri, 19 Aug 2011 15:39:26 GMT
<
* Connection #0 to host storage101.
* Re-using existing connection! (#0) with host storage101.
* Connected to storage101.
> HEAD /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 404 Not Found
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx10edbc85d8fb4
< Date: Fri, 19 Aug 2011 15:39:26 GMT
<
* Connection #0 to host storage101.
* Re-using existing connection! (#0) with host storage101.
* Connected to storage101.
> PUT /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Object-Meta-Link: test
Content-
Content-Encoding: gzip
X-Object-Manifest: test/manifest-
ETag: 5058f1af8388633
Content-Type: text/plain
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
Content-Length: 1
Expect: 100-continue
< HTTP/1.1 100 Continue
< HTTP/1.1 201 Created
< Content-Length: 118
< Content-Type: text/html; charset=UTF-8
< Etag: 5058f1af8388633
< Last-Modified: Fri, 19 Aug 2011 15:39:26 GMT
< X-Trans-Id: txf18c33510a3d4
< Date: Fri, 19 Aug 2011 15:39:26 GMT
<
* Connection #0 to host storage101.
* Re-using existing connection! (#0) with host storage101.
* Connected to storage101.
> HEAD /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 200 OK
< X-Object-Meta-Data: test
< Content-
< Content-Encoding: gzip
< Last-Modified: Fri, 19 Aug 2011 15:39:26 GMT
< Etag: 38962e22d2a9e97
< Accept-Ranges: bytes
< Content-Length: 24
< Content-Type: application/x-gzip
< X-Trans-Id: tx3ee2495962b04
< Date: Fri, 19 Aug 2011 15:39:26 GMT
<
* Connection #0 to host storage101.
* Re-using existing connection! (#0) with host storage101.
* Connected to storage101.
> HEAD /v1/MossoCloudF
Host: storage101.
Accept: */*
X-Auth-Token: <hidden>
User-Agent: PHP-CloudFiles/
< HTTP/1.1 200 OK
< X-Object-Manifest: test/manifest-
< Content-Type: text/plain
< X-Object-Meta-Link: test
< Etag: 8bfb9e5e3baa089
< Content-Length: 24
< Last-Modified: Fri, 19 Aug 2011 15:39:26 GMT
< Accept-Ranges: bytes
< X-Trans-Id: tx8b2a98052fd84
< Date: Fri, 19 Aug 2011 15:39:27 GMT
<
* Connection #0 to host storage101.
-------
http://
-------
* Closing connection #0
* Closing connection #0
* Closing connection #0
mss@TreeHouse:
GET http://
User-Agent: lwp-request/5.834 libwww-perl/5.837
GET http://
Cache-Control: public, max-age=86400
Connection: close
Date: Fri, 19 Aug 2011 15:42:12 GMT
ETag: 38962e22d2a9e97
Server: nginx/0.7.65
Content-Length: 4
Content-Type: application/x-gzip
Expires: Sat, 20 Aug 2011 15:42:12 GMT
Last-Modified: Fri, 19 Aug 2011 15:39:26 GMT
Client-Date: Fri, 19 Aug 2011 15:42:12 GMT
Client-Peer: 195.95.193.9:80
Client-
Content-
X-Object-Meta-Data: test
X-Trans-Id: txa5ecc909568f4
foo
mss@TreeHouse:
GET http://
User-Agent: lwp-request/5.834 libwww-perl/5.837
GET http://
Cache-Control: public, max-age=85217
Connection: close
Date: Fri, 19 Aug 2011 15:42:16 GMT
Accept-Ranges: bytes
ETag: 8bfb9e5e3baa089
Server: nginx/0.7.65
Content-Length: 24
Content-Type: text/plain; charset=UTF-8
Expires: Sat, 20 Aug 2011 15:22:33 GMT
Last-Modified: Fri, 19 Aug 2011 15:09:14 GMT
Client-Date: Fri, 19 Aug 2011 15:42:15 GMT
Client-Peer: 195.95.193.16:80
Client-
X-Object-Manifest: test/manifest-
X-Object-Meta-Link: test
X-Trans-Id: tx403f9813ea774
<binary data omitted>
Related branches
- Greg Lange (community): Approve
- gholt (community): Approve
-
Diff: 73 lines (+8/-18)2 files modifiedswift/proxy/server.py (+3/-16)
test/unit/proxy/test_server.py (+5/-2)
| Changed in swift: | |
| status: | New → Confirmed |
| importance: | Undecided → High |
| Changed in swift: | |
| assignee: | nobody → John Dickinson (notmyname) |
| milestone: | none → 1.4.3 |
| status: | Confirmed → In Progress |
| Changed in swift: | |
| status: | In Progress → Fix Committed |
| Changed in swift: | |
| status: | Fix Committed → Fix Released |
