I found python-botocore 1.16.19+repack-1ubuntu0.16.04.2 unable to make S3 requests. Minimal reproducer:
import os
import boto3
os.environ["AWS_ACCESS_KEY_ID"] = "ABCABCABCABCABCABCAB"
os.environ["AWS_SECRET_ACCESS_KEY"] = "YWJjYWJjYWJjYWJjYWJjYWJjYWJjYWJjYWJhYmMK"
os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
boto3.client("s3").list_objects(Bucket="abc")
The error I get with python-botocore 1.16.19+repack-1ubuntu0.16.04.2 from xenial-updates/universe is:
$ python repro.py
Traceback (most recent call last):
File "repro.py", line 6, in <module>
boto3.client("s3").list_objects(Bucket="abc")
File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 622, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 641, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 269, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
botocore.exceptions.HTTPClientError: An HTTP Client raised and unhandled exception: name must be a byte string
Downgrading it to python-botocore 1.3.23-1 from xenial/universe solves the problem (the error changes to "The AWS Access Key Id you provided does not exist in our records" but this is expected).
The error exception "name must be a byte string" originates from pyopnessl (https://github.com/pyca/pyopenssl/issues/515) so my understanding is that the latest version of botocore in xenial-updates is incompatible with xenial's ssl libraries.
Please find attached a patch which fixes the backport of python-botocore 1.16.19 to Ubuntu 16.04. The patch contains a more detailed description of what's exactly broken with 1.16.19+ repack- 1ubuntu0. 16.04.2.