ranged requests on large objects have incorrect behaviour when manifest file is not 0 bytes.

[Andrew Hale]

Ranged requests on large objects have incorrect behaviour when Manifest file is not 0 bytes.
So you can see the files used in the examples below, this is what the contents and containers look like:

---
$ for x in `seq -w 1 10`;do echo ${x}_AAAAAA > ABC_${x}.txt; done
$ for x in `seq -w 1 10`;do echo ${x}_aaaaaa > abc_${x}.txt; done
$ echo ABC > manifest-ABC.txt
$ echo abc > manifest-abc.txt
$ touch meta-manifest.txt
$ dd if=/dev/zero bs=1 count=1000 of=bigmanifest.txt
$ ls -l ABC_01.txt manifest-ABC.txt bigmanifest.txt
-rw-r--r-- 1 andy staff 10 30 Mar 16:19 ABC_01.txt
-rw-r--r-- 1 andy staff 1000 30 Mar 16:02 bigmanifest.txt
-rw-r--r-- 1 andy staff 4 30 Mar 15:58 manifest-ABC.txt

$ swift list test
abc_01.txt
abc_02.txt
abc_03.txt
abc_04.txt
abc_05.txt
abc_06.txt
abc_07.txt
abc_08.txt
abc_09.txt
abc_10.txt
bigmanifest.txt
manifest-abc.txt

$ curl -D - -X POST -H "${KEY}" -H 'X-Object-Manifest: test/abc_' ${URL}/test/manifest-abc.txt
$ curl -D - -X POST -H "${KEY}" -H 'X-Object-Manifest: test/abc_' ${URL}/test/bigmanifest.txt

---

So manifest-abc.txt is a 4 byte files pointing at 10x 10 byte segments and bigmanifest is a 1000 byte file pointing the same 10x 10 byte objects.

When making ranged requests, if the size of the manifest file is non zero and the range includes bytes which are also present in the manifest then the full object is returned with a 200 code rather than 206, with the full, correct content length and with the total available byte range in the header reflecting the size of the manifest object rather than the sum of the segments.

I don't know whether the proxy should just reject non 0 byte manifest files, though existing deployments may already have content like this.

In these there is a 4 byte manifest file for 100 bytes of segmented files. Range reqs including the first 4 bytes return the full object.

$ curl -H "Range: bytes=1-5" -D - -X GET -H "${KEY}" ${URL}/test/manifest-abc.txt
HTTP/1.1 200 OK
Content-Range: bytes 1-3/4
X-Object-Manifest: test/abc_
Content-Type: text/plain
Content-Length: 100

$ curl -H "Range: bytes=7-9" -D - -X GET -H "${KEY}" ${URL}/test/manifest-abc.txt
HTTP/1.1 206 Partial Content
Content-Length: 3
Content-Range: bytes 7-9/100
X-Object-Manifest: test/abc_
Content-Type: text/plain

$ curl -H "Range: bytes=4-9" -D - -X GET -H "${KEY}" ${URL}/test/manifest-abc.txt
HTTP/1.1 206 Partial Content
Content-Length: 6
Content-Range: bytes 4-9/100
X-Object-Manifest: test/abc_
Content-Type: text/plain

$ curl -H "Range: bytes=3-9" -D - -X GET -H "${KEY}" ${URL}/test/manifest-abc.txt
HTTP/1.1 200 OK
Content-Range: bytes 3-3/4
X-Object-Manifest: test/abc_
Content-Type: text/plain
Content-Length: 100

Here, the manifest file is larger than the sum of the segments, ranged requests always return a 200 and full large object.

$ curl -H "Range: bytes=3-9" -D - -X GET -H "${KEY}" ${URL}/test/bigmanifest.txt
HTTP/1.1 200 OK
Content-Range: bytes 3-9/1000
X-Object-Manifest: test/abc_
Content-Type: text/plain
Content-Length: 100

$ curl -H "Range: bytes=80-90" -D - -X GET -H "${KEY}" ${URL}/test/bigmanifest.txt
HTTP/1.1 200 OK
Content-Range: bytes 80-90/1000
X-Object-Manifest: test/abc_
Content-Type: text/plain
Content-Length: 100

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/8872

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/8872
Committed: http://github.com/openstack/swift/commit/49a9cc7e68c1558b1fabb71ebdf6c6c70a9b6a3e
Submitter: Jenkins
Branch: master

commit 49a9cc7e68c1558b1fabb71ebdf6c6c70a9b6a3e
Author: Iryoung Jeong <email address hidden>
Date: Sat Jun 23 05:19:00 2012 +0900

    Made ranged requests on large objects working correctly when size of manifest file is not 0 byte.

    Fixed bug 969411

    Ranged GET request checked wheter the object is large objects only when response is 416 Requested Range Not Satisfiable.

    All ranged GET requests should check the object is large object or not because size of the manifest file can be larger than 0.

    Change-Id: Ia3632252d6879a773be881149c10ce0f5ffbcd82