AttributeError when a header contains only non-printable chars

[Yuxin Wang]

When we tested using ceph/s3-tests, for some test cases, the request headers contain only non-printable chars and it would raise 'NoneType' object has no attribute 'split' error.

Cases include:

s3tests.functional.test_headers.test_object_create_bad_md5_unreadable

s3tests.functional.test_headers.test_object_create_bad_contenttype_unreadable

s3tests.functional.test_headers.test_object_create_bad_ua_unreadable_aws4

s3tests.functional.test_headers.test_object_create_bad_date_unreadable_aws4

s3tests.functional.test_headers.test_bucket_create_bad_ua_unreadable_aws4

s3tests.functional.test_headers.test_bucket_create_bad_date_unreadable_aws4

Versions we are using:

swift3: 1.12

ceph/s3-tests: f318716c6

Debug info and traceback:

boto: DEBUG: path=/testbucket/
boto: DEBUG: auth_path=/testbucket/
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /testbucket/
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: s3.US.test.com
boto: DEBUG: Port: 80
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: Final headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Content-Length': '0', 'X-Amz-Date': '20180416T082511Z', 'Host': 's3.US.amazonaws.com', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test/20180416/US/s3/aws4_request,SignedHeaders=host;user-agent;x-amz-content-sha256;x-amz-date,Signature=3911fc375ff88de82c58f51180494399eaac73a7e9d9aa35917c5e5a53a6f145', 'User-Agent': '\x07'}
boto: DEBUG: Response headers: [('x-amz-id-2', 'txc580336b329845189bce8-005ad45de5'), ('transfer-encoding', 'chunked'), ('x-amz-request-id', 'txc580336b329845189bce8-005ad45de5'), ('x-trans-id', 'txc580336b329845189bce8-005ad45de5'), ('date', 'Mon, 16 Apr 2018 08:25:09 GMT'), ('content-type', 'application/xml'), ('x-openstack-request-id', 'txc580336b329845189bce8-005ad45de5')]
boto: DEBUG: Received 500 response. Retrying in 0.7 seconds

Apr 16 16:25:20 test_161 swift3: 'NoneType' object has no attribute 'split':
Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/swift3/middleware.py", line 79, in __call__
        req = req_class(env, self.app, self.slo_enabled)
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 1281, in __init__
        super(S3AclRequest, self).__init__(env, slo_enabled)
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 414, in __init__
        self.string_to_sign = self._string_to_sign()
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 374, in _string_to_sign
        sha256(self._canonical_request()).hexdigest()])
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 341, in _canonical_request
        headers_to_sign = self._headers_to_sign()
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 281, in _headers_to_sign
        for (k, v) in six.iteritems(self.headers))
    File "/usr/lib/python2.7/site-packages/swift3/request.py", line 281, in <genexpr>
        for (k, v) in six.iteritems(self.headers))
    AttributeError: 'NoneType' object has no attribute 'split' (txn: tx8427a7a863e04fae9bdbc-005ad45df0)

[Yuxin Wang (chhyx2008)]

After _header_strip the header contains only non-printable, it returns None, and None can't split.

https://github.com/openstack/swift3/blob/1.12/swift3/request.py#L280

[Tim Burke (1-tim-z)]

Also affects s3api.