| 2016-01-27 14:24:17 |
Alistair Coles |
description |
When an SLO manifest A in a versioned container is overwritten by object B, the manifest A is resolved and the assembled large object is copied to the versions container. The original manifest is lost. Then when the object B in the versioned container is deleted, it is replaced with the large object, not the original manifest.
To reproduce:
#create containers, segment and manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions" -X PUT
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_segments" -X PUT
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1" -X PUT -H 'X-Versions-Location: c1_versions'
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_segments/4MB_file" -X PUT -T 4MB_file
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart-manifest=put" -X PUT -T slo_manifest
# sanity check, the versions container is empty and the manifest is a manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions?format=json"
[]%
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Last-Modified: Fri, 22 Jan 2016 12:16:41 GMT
Connection: close
Etag: "a42db41ea9e4a932e67a57ece7cf361b"
X-Timestamp: 1453465000.92552
Date: Fri, 22 Jan 2016 12:17:56 GMT
Content-Type: application/octet-stream
X-Static-Large-Object: True
X-Trans-Id: tx4fb306cefd4b4ec089ccf-0056a21df4
# now overwrite the manifest, it doesn't matter what the content is...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart-manifest=put" -X PUT -T slo_manifest
# oops, now the versions container has 4MB file
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions?format=json"[{"hash": "b5cfa9d6c8febd618f91ac2843d50a1c", "last_modified": "2016-01-22T12:29:45.369080", "bytes": 4194304, "name": "00cslo_manifest/1453465138.03144", "content_type": "application/octet-stream"}]%
# and its not just the listing telling us that when really its a manifest, it really is not a manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions/00cslo_manifest/1453465138.03144?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Accept-Ranges: bytes
Last-Modified: Fri, 22 Jan 2016 12:29:46 GMT
Etag: b5cfa9d6c8febd618f91ac2843d50a1c
X-Timestamp: 1453465785.36908
Content-Type: application/octet-stream
X-Trans-Id: tx26f122fcaa2049b49e9d7-0056a220ff
Date: Fri, 22 Jan 2016 12:30:55 GMT
# now delete the object in the versioned container...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest" -X DELETE
# and oops, we now have alare object, not the manifest, copied back
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Accept-Ranges: bytes
Last-Modified: Fri, 22 Jan 2016 12:31:09 GMT
Etag: b5cfa9d6c8febd618f91ac2843d50a1c
X-Timestamp: 1453465868.72903
Content-Type: application/octet-stream
X-Trans-Id: txd37023de79194f558902e-0056a22116
Date: Fri, 22 Jan 2016 12:31:18 GMT |
Updated description:
When versioned_writes is not explicitly configured in the proxy pipeline and *LO middlewares *are* configured, then the versioned_writes middleware gets inserted to the left of DLO, which is incorrect and causes the following incorrect behaviour to manifest...
Original description:
When an SLO manifest A in a versioned container is overwritten by object B, the manifest A is resolved and the assembled large object is copied to the versions container. The original manifest is lost. Then when the object B in the versioned container is deleted, it is replaced with the large object, not the original manifest.
To reproduce:
#create containers, segment and manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions" -X PUT
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_segments" -X PUT
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1" -X PUT -H 'X-Versions-Location: c1_versions'
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_segments/4MB_file" -X PUT -T 4MB_file
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart-manifest=put" -X PUT -T slo_manifest
# sanity check, the versions container is empty and the manifest is a manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions?format=json"
[]%
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Last-Modified: Fri, 22 Jan 2016 12:16:41 GMT
Connection: close
Etag: "a42db41ea9e4a932e67a57ece7cf361b"
X-Timestamp: 1453465000.92552
Date: Fri, 22 Jan 2016 12:17:56 GMT
Content-Type: application/octet-stream
X-Static-Large-Object: True
X-Trans-Id: tx4fb306cefd4b4ec089ccf-0056a21df4
# now overwrite the manifest, it doesn't matter what the content is...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart-manifest=put" -X PUT -T slo_manifest
# oops, now the versions container has 4MB file
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions?format=json"[{"hash": "b5cfa9d6c8febd618f91ac2843d50a1c", "last_modified": "2016-01-22T12:29:45.369080", "bytes": 4194304, "name": "00cslo_manifest/1453465138.03144", "content_type": "application/octet-stream"}]%
# and its not just the listing telling us that when really its a manifest, it really is not a manifest...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1_versions/00cslo_manifest/1453465138.03144?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Accept-Ranges: bytes
Last-Modified: Fri, 22 Jan 2016 12:29:46 GMT
Etag: b5cfa9d6c8febd618f91ac2843d50a1c
X-Timestamp: 1453465785.36908
Content-Type: application/octet-stream
X-Trans-Id: tx26f122fcaa2049b49e9d7-0056a220ff
Date: Fri, 22 Jan 2016 12:30:55 GMT
# now delete the object in the versioned container...
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest" -X DELETE
# and oops, we now have alare object, not the manifest, copied back
% curl -H 'X-Auth-Token: AUTH_tk5a7c8aa2656847769335c3809ab62c6b' "http://localhost:8080/v1/AUTH_test/c1/slo_manifest?multipart_manifest=get" -I
HTTP/1.1 200 OK
Content-Length: 4194304
Accept-Ranges: bytes
Last-Modified: Fri, 22 Jan 2016 12:31:09 GMT
Etag: b5cfa9d6c8febd618f91ac2843d50a1c
X-Timestamp: 1453465868.72903
Content-Type: application/octet-stream
X-Trans-Id: txd37023de79194f558902e-0056a22116
Date: Fri, 22 Jan 2016 12:31:18 GMT |
|
