In PUT Object Copy, 'x-amz-metadata-directive: REPLACE' does not work.

[Masaki Tsukuda]

When I specify 'x-amz-metadata-directive: REPLACE' in PUT Object Copy,
I expect that metada is not copied.
However, metadata is copied in Swift3.

@ swift3
$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/ -X PUT
$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/src -X PUT -T 1kb.file -H 'x-amz-meta-src: src'
$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/src -X HEAD -I
HTTP/1.1 200 OK
Content-Length: 1024
x-amz-id-2: tx746fb065c28441bb99bc7-00550a365e
Last-Modified: Thu, 19 Mar 2015 02:37:01 GMT
ETag: "44f488ecb0673735db68676c31435569"
x-amz-meta-src: src
x-amz-request-id: tx746fb065c28441bb99bc7-00550a365e
Content-Type: application/octet-stream
X-Trans-Id: tx746fb065c28441bb99bc7-00550a365e
Date: Thu, 19 Mar 2015 02:37:18 GMT

$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/dst -X PUT -H 'x-amz-copy-source: /test-directive/src' -H 'x-amz-metadata-directive: REPLACE'
<?xml version='1.0' encoding='UTF-8'?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <LastModified>2015-03-19T02:38:20.000Z</LastModified>
    <ETag>"44f488ecb0673735db68676c31435569"</ETag>
</CopyObjectResult>

$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/dst -X HEAD
HTTP/1.1 200 OK
Content-Length: 1024
x-amz-id-2: tx3c15c21e428a445ead2e0-00550a36af
Last-Modified: Thu, 19 Mar 2015 02:38:20 GMT
ETag: "44f488ecb0673735db68676c31435569"
x-amz-meta-src: src
x-amz-request-id: tx3c15c21e428a445ead2e0-00550a36af
Content-Type: application/octet-stream
X-Trans-Id: tx3c15c21e428a445ead2e0-00550a36af
Date: Thu, 19 Mar 2015 02:38:39 GMT

@AWS S3
$ ./s3curl.pl --id aws -- https://s3.amazonaws.com/test-directive -X PUT
$ ./s3curl.pl --id aws -- https://s3.amazonaws.com/test-directive/src -X PUT -T 1kb.file -H 'x-amz-meta-src: src'
$ ./s3curl.pl --id aws -- https://s3.amazonaws.com/test-directive/src -X HEAD
HTTP/1.1 200 OK
x-amz-id-2: WV48YhMTWKi2mGHPjQdXlCNcj9XTRbctwKn87BjetLrW2+UdtH1/5G8EIIJZ3JdJ
x-amz-request-id: 38040BF125E38834
Date: Thu, 19 Mar 2015 02:42:29 GMT
x-amz-meta-src: src
Last-Modified: Thu, 19 Mar 2015 02:41:59 GMT
ETag: "44f488ecb0673735db68676c31435569"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 1024
Server: AmazonS3

$ ./s3curl.pl --id aws -- https://s3.amazonaws.com/test-directive/dst -X PUT -H 'x-amz-copy-source: /test-directive/src' -H 'x-amz-metadata-directive: REPLACE'
<?xml version="1.0" encoding="UTF-8"?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <LastModified>2015-03-19T02:43:27.000Z</LastModified>
    <ETag>&quot;44f488ecb0673735db68676c31435569&quot;</ETag>
</CopyObjectResult>

$ ./s3curl.pl --id aws -- https://s3.amazonaws.com/test-directive/dst -X HEAD
HTTP/1.1 200 OK
x-amz-id-2: qJ0CZR0wUGULMX3kmU8q4idMZ9MYDYM+I7WB9CfgLmKvvtxtTJW4FmivmUO6xyBv
x-amz-request-id: EAB3FE18DD19EE81
Date: Thu, 19 Mar 2015 02:43:45 GMT
Last-Modified: Thu, 19 Mar 2015 02:43:27 GMT
ETag: "44f488ecb0673735db68676c31435569"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 1024
Server: AmazonS3

[Kota Tsuyuzaki (tsuyuzaki-kota)]

Please specify the meta data you want to replace at the copy request.

S3 api reference[1] shows the description about x-amz-metadata-directive as follows:
If replaced, all original metadata is replaced by the metadata you specify.

1: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html

[Masaki Tsukuda (tsukuda-masaki)]

I tried again with specify metadata in PUT Object Copy.
As a result source metadata is remaining.

$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/ -X PUT
$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/src -X PUT -T testFile -s -H 'x-amz-meta-src: src'
$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/src -X HEAD -I
HTTP/1.1 200 OK
Content-Length: 11
x-amz-id-2: txb51acb733b634611b298b-0055128ab0
Last-Modified: Wed, 25 Mar 2015 10:14:23 GMT
ETag: "7c12772809c1c0c3deda6103b10fdfa0"
x-amz-meta-src: src
x-amz-request-id: txb51acb733b634611b298b-0055128ab0
Content-Type: application/octet-stream
X-Trans-Id: txb51acb733b634611b298b-0055128ab0
Date: Wed, 25 Mar 2015 10:15:12 GMT

$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/dst -X PUT -H 'x-amz-copy-source:/test-directive/src' -H 'x-amz-meta-dst: dst' -H 'x-amz-metadata-directive: REPLACE'
<?xml version='1.0' encoding='UTF-8'?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <LastModified>2015-03-25T10:16:25.000Z</LastModified>
  <ETag>"7c12772809c1c0c3deda6103b10fdfa0"</ETag>
</CopyObjectResult>

$ ./s3curl.pl --id test -- http://localhost:8080/test-directive/dst -X HEAD -I
HTTP/1.1 200 OK
Content-Length: 11
x-amz-id-2: txf2147110b4d94edcaa3d1-0055128b0c
x-amz-meta-dst: dst
Last-Modified: Wed, 25 Mar 2015 10:16:25 GMT
ETag: "7c12772809c1c0c3deda6103b10fdfa0"
x-amz-meta-src: src
x-amz-request-id: txf2147110b4d94edcaa3d1-0055128b0c
Content-Type: application/octet-stream
X-Trans-Id: txf2147110b4d94edcaa3d1-0055128b0c
Date: Wed, 25 Mar 2015 10:16:45 GMT

[Kota Tsuyuzaki (tsuyuzaki-kota)]

x-amz-metadata-directive: REPLACE used for overwriting the metadata specified.

i.e. you must set x--amz-meta-src: dst on the second request, otherwise you cannot "REPLACE" the value.

[Masaki Tsukuda (tsukuda-masaki)]

> If replaced, all original metadata is replaced by the metadata you specify.

all original metada should be replaced.

[Kota Tsuyuzaki (tsuyuzaki-kota)]

"by the metadata you specify."?

You have to specify to replace the metadata.

[Masaki Tsukuda (tsukuda-masaki)]

Please confirm the results of the run in S3.

[Kota Tsuyuzaki (tsuyuzaki-kota)]

We need confirmation.

[Darryl Tam (dtam)]

Right now, x-amz-metadata-directive: has no effect on a copy operation.

In all cases (REPLACE, COPY or missing), custom metadata specified is applied to the copied object, while non-conflicting metadata remains.

However, the behaviour should be that under REPLACE: Only the custom metadata should be applied, when COPYING (or when directive is missing), the existing custom metadata is copied, while any specified metadata is ignored.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tim Burke (<email address hidden>) on branch: master
Review: https://review.openstack.org/223857
Reason: Proposed against s3api as https://review.openstack.org/#/c/575824/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/swift 2.20.0 release.