botocore API missmatch with python3-requests

Bug #1711737 reported by Tully
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python-botocore (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

The upstream package integrates a newer version of the requests python library. In the debian package it is modified to use the debian packages python3-requests. However it's not modified to use the older requests API fully.

At least the bucket.upload_file method is broken in the debian packaging. I ticketed it upstream https://github.com/boto/botocore/issues/1266 but since it's a packaging issue I'm refiling it here.

Here's my script failing with the call to requests.

```
$ python3 run_test.py
Uploading results for test: test_upload_20170817142448 from /tmp/output
Uploading /tmp/output/ros.bag to test_upload_20170817142448/ros.bag
Uploading /tmp/output/gazebo.log to test_upload_20170817142448/gazebo.log
Traceback (most recent call last):
  File "run_test.py", line 47, in <module>
    upload_to_s3(b, test_name, results_dir)
  File "run_test.py", line 14, in upload_to_s3
    bucket.upload_file(full_f, bucket_key)
  File "/usr/lib/python3/dist-packages/boto3/s3/inject.py", line 110, in bucket_upload_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/usr/lib/python3/dist-packages/boto3/s3/inject.py", line 71, in upload_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/lib/python3/dist-packages/boto3/s3/transfer.py", line 629, in upload_file
    self._put_object(filename, bucket, key, callback, extra_args)
  File "/usr/lib/python3/dist-packages/boto3/s3/transfer.py", line 639, in _put_object
    **extra_args)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 526, in _make_api_call
    operation_model, request_dict)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 141, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 166, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 151, in create_request
    prepared_request = self.prepare_request(request)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 162, in prepare_request
    return request.prepare()
  File "/usr/lib/python3/dist-packages/botocore/awsrequest.py", line 369, in prepare
    p.prepare_body(self.data, self.files)
  File "/usr/lib/python3/dist-packages/botocore/awsrequest.py", line 440, in prepare_body
    data.seek(0, 2)
TypeError: seek() takes 2 positional arguments but 3 were given
```

I have a local workaround that doesn't call `upload_file` but just calls `put_object` instead that avoids the problem.
```
            print('Uploading %s to %s' % (full_f, bucket_key))
            if os.stat(full_f).st_size:
                bucket.upload_file(full_f, bucket_key)
            else:
                print("Empty file fallback behavior")
                bucket.put_object(
                    Key=bucket_key,
                    Body=''
                )
```

And here's it passing with the workaround above.
```
$ python3 run_test.py
Uploading results for test: test_upload_20170817144559 from /tmp/output
Uploading /tmp/output/ros.bag to test_upload_20170817144559/ros.bag
Uploading /tmp/output/gazebo.log to test_upload_20170817144559/gazebo.log
Empty file fallback behavior
Uploading /tmp/output/test_results/results1.xml to test_upload_20170817144559/results1.xml
Empty file fallback behavior
Uploading /tmp/output/test_results/results2.xml to test_upload_20170817144559/results2.xml
Empty file fallback behavior
```

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-botocore (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.