S3browser - AWS V4 signature
Trying to restore an object's older version as the current version fails due to 403 error.
[W] [9/16/2020 10:09:53] SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method. TaskID: 62753
[E] [9/16/2020 10:09:53] Error occurred during Copying bigcontainer-0-0/errorlog.html to bigcontainer-0-0/errorlog.html SignatureDoesNotMatch TaskID: 62753
The signature mismatch is because s3browser is adding some of the headers returned by an earlier HEAD on the object referred to as x-amz-copy-source. This should be fine but the headers X-Trans-Id and X-Openstack-Request-Id are not in lower case, whereas the signed headers in aws v4 signature are supposed to be lower case.
{'x-amz-metadata-directive', 'x-amz-acl', 'x-openstack-request-id', 'user-agent', 'x-amz-content-sha256', 'content-type', 'x-amz-meta-s3b-last-modified', 'x-amz-date', 'host', 'x-amz-meta-sha256', 'x-amz-storage-class', 'x-amz-copy-source', 'x-trans-id'}
Looking into the catch_errors.py middleware[1], we are explicitly adding the X-Trans-Id[1] and X-Openstack-Request-Id[2] in the response in pascal case instead of the lower case.
Changing them to lower case allows restore to go through.
Wireshark traces:
PUT /bigcontainer-0-0/errorlog.html HTTP/1.1
User-Agent: S3 Browser 8.9.7 https://s3browser.com
Content-Type: text/html
Authorization: AWS4-HMAC-SHA256 Credential=2ea2a7883ea4477ab5aec87b8aee6f95/20200916/us-east-1/s3/aws4_request,SignedHeaders=content-type;host;user-agent;x-openstack-request-id;x-trans-id;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-meta-s3b-last-modified;x-amz-meta-sha256;x-amz-metadata-directive;x-amz-storage-class, Signature=bdf0c42da2df3124628ee47019bb8b8ba51c71f69e0ac48f9a4349ca117e8e90
X-Openstack-Request-Id: tx0a2aa463f64f4aaea4d7d-005f6202de
X-Trans-Id: tx0a2aa463f64f4aaea4d7d-005f6202de
x-amz-acl: private
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-copy-source: /bigcontainer-0-0/errorlog.html?versionId=NGFjMzliOTYtZGZmNC00ZjU3LTk4MDYtMzIwNWYzNGY4NzAx
x-amz-date: 20200916T121737Z
x-amz-meta-s3b-last-modified: 20130613T075420Z
x-amz-meta-sha256: dc96e165f4a2da5b7454bf6de0bdf53ef1e15f6c8f3fd8fc929dacad5efd4078
x-amz-metadata-directive: COPY
x-amz-storage-class: STANDARD
Host: xxxxxxxxxxxxxxxxxxxxxx
Content-Length: 0
HTTP/1.1 403 Forbidden
Content-Type: application/xml
x-amz-id-2: txdf93ddf985d24e36a0fc9-005f6202df
x-amz-request-id: txdf93ddf985d24e36a0fc9-005f6202df
X-Trans-Id: txdf93ddf985d24e36a0fc9-005f6202df
X-Openstack-Request-Id: txdf93ddf985d24e36a0fc9-005f6202df
Date: Wed, 16 Sep 2020 12:19:43 GMT
Transfer-Encoding: chunked
<?xml version='1.0' encoding='UTF-8'?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><RequestId>txdf93ddf985d24e36a0fc9-005f6202df</RequestId><AWSAccessKeyId>2ea2a7883ea4477ab5aec87b8aee6f95</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20200916T121737Z
20200916/us-east-1/s3/aws4_request
aa10a6ca56ba52291f2b0cca8e7b92918286cb0939cd66d3cee3ff6064c409a7</StringToSign><SignatureProvided>bdf0c42da2df3124628ee47019bb8b8ba51c71f69e0ac48f9a4349ca117e8e90</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 30 30 39 31 36 54 31 32 31 37 33 37 5a 0a 32 30 32 30 30 39 31 36 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 61 61 31 30 61 36 63 61 35 36 62 61 35 32 32 39 31 66 32 62 30 63 63 61 38 65 37 62 39 32 39 31 38 32 38 36 63 62 30 39 33 39 63 64 36 36 64 33 63 65 65 33 66 66 36 30 36 34 63 34 30 39 61 37</StringToSignBytes><CanonicalRequest>PUT
/bigcontainer-0-0/errorlog.html
content-type:text/html
host:xxxxxxxxxxxxxxxxxxxxxx
user-agent:S3 Browser 8.9.7 https://s3browser.com
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-copy-source:/bigcontainer-0-0/errorlog.html?versionId=NGFjMzliOTYtZGZmNC00ZjU3LTk4MDYtMzIwNWYzNGY4NzAx
x-amz-date:20200916T121737Z
x-amz-meta-s3b-last-modified:20130613T075420Z
x-amz-meta-sha256:dc96e165f4a2da5b7454bf6de0bdf53ef1e15f6c8f3fd8fc929dacad5efd4078
x-amz-metadata-directive:COPY
x-amz-storage-class:STANDARD
x-openstack-request-id:tx0a2aa463f64f4aaea4d7d-005f6202de
x-trans-id:tx0a2aa463f64f4aaea4d7d-005f6202de
content-type;host;user-agent;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-meta-s3b-last-modified;x-amz-meta-sha256;x-amz-metadata-directive;x-amz-storage-class;x-openstack-request-id;x-trans-id
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 62 69 67 63 6f 6e 74 61 69 6e 65 72 2d 30 2d 30 2f 65 72 72 6f 72 6c 6f 67 2e 68 74 6d 6c 0a 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 74 65 78 74 2f 68 74 6d 6c 0a 68 6f 73 74 3a 76 73 61 2d 30 30 30 30 31 30 66 39 2d 7a 61 64 61 72 61 2d 64 65 76 32 2e 7a 61 64 61 72 61 7a 69 6f 73 2e 63 6f 6d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 53 33 20 42 72 6f 77 73 65 72 20 38 2e 39 2e 37 20 68 74 74 70 73 3a 2f 2f 73 33 62 72 6f 77 73 65 72 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 72 69 76 61 74 65 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3a 2f 62 69 67 63 6f 6e 74 61 69 6e 65 72 2d 30 2d 30 2f 65 72 72 6f 72 6c 6f 67 2e 68 74 6d 6c 3f 76 65 72 73 69 6f 6e 49 64 3d 4e 47 46 6a 4d 7a 6c 69 4f 54 59 74 5a 47 5a 6d 4e 43 30 30 5a 6a 55 33 4c 54 6b 34 4d 44 59 74 4d 7a 49 77 4e 57 59 7a 4e 47 59 34 4e 7a 41 78 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 32 30 30 39 31 36 54 31 32 31 37 33 37 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 73 33 62 2d 6c 61 73 74 2d 6d 6f 64 69 66 69 65 64 3a 32 30 31 33 30 36 31 33 54 30 37 35 34 32 30 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 73 68 61 32 35 36 3a 64 63 39 36 65 31 36 35 66 34 61 32 64 61 35 62 37 34 35 34 62 66 36 64 65 30 62 64 66 35 33 65 66 31 65 31 35 66 36 63 38 66 33 66 64 38 66 63 39 32 39 64 61 63 61 64 35 65 66 64 34 30 37 38 0a 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3a 43 4f 50 59 0a 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 3a 53 54 41 4e 44 41 52 44 0a 78 2d 6f 70 65 6e 73 74 61 63 6b 2d 72 65 71 75 65 73 74 2d 69 64 3a 74 78 30 61 32 61 61 34 36 33 66 36 34 66 34 61 61 65 61 34 64 37 64 2d 30 30 35 66 36 32 30 32 64 65 0a 78 2d 74 72 61 6e 73 2d 69 64 3a 74 78 30 61 32 61 61 34 36 33 66 36 34 66 34 61 61 65 61 34 64 37 64 2d 30 30 35 66 36 32 30 32 64 65 0a 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3b 68 6f 73 74 3b 75 73 65 72 2d 61 67 65 6e 74 3b 78 2d 61 6d 7a 2d 61 63 6c 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 73 33 62 2d 6c 61 73 74 2d 6d 6f 64 69 66 69 65 64 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3b 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 3b 78 2d 6f 70 65 6e 73 74 61 63 6b 2d 72 65 71 75 65 73 74 2d 69 64 3b 78 2d 74 72 61 6e 73 2d 69 64 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes></Error>
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py#L120
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py#L121
