S3 upload of empty files fails

[Lienhart Woitok]

Trying to use awscli to upload files to S3 fails on empty files. The developers of awscli claim that this is a bug in ubuntus packaging, therefore I'm reporting it here since noone seems to have done before.

Details can be found in the following github issue:
https://github.com/aws/aws-cli/issues/2403

I'm currently affected by this bug and can test a new version, but I can not give any more insights into the issue or what needs to be done.

[Christian Ehrhardt  (paelzer)]

Thanks Lienhart for the report, I must admit I don't know the backgrounds what might have happened here in regard to the packaging that was referred to in the discussions of the issue you linked.

Subscribing Kamal as he worked on the packaging to get his expertise on this.

[Kamal Mostafa (kamalmostafa)]

This regression occurs in Xenial/Yakkety (1.11.13-1ubuntu1~16.*.0) and also in Zesty/Artful (1.11.44-1):

$ aws --version
aws-cli/1.11.44 Python/3.5.3 Linux/4.10.0-19-generic botocore/1.5.7

$ touch emptyfile

$ aws s3 cp emptyfile s3://kamal-test-1
upload failed: ./emptyfile to s3://kamal-test-1/emptyfile seek() takes 2 positional arguments but 3 were given

[Kamal Mostafa (kamalmostafa)]

The failure appears to be due to the incomplete implementation of seek() -- missing the optional 'whence' argument -- in python-s3transfer's ReadFileChunk method.

The attached patch to python-s3transfer fixes it in my testing. Before submitting the patch to the Ubuntu and upstream, I'd like an additional verification that it resolves the problem. Here's a PPA for Xenial which provides python-s3tranfer with the patch applied:

https://launchpad.net/~kamalmostafa/+archive/ubuntu/lp1696800

Please confirm that this PPA (installed along with the stock Ubuntu awscli package) fixes the problem for you.

[Lienhart Woitok (lienhart-woitok)]

Hello Kamal,

I can confirm that your proposed fix actually fixes the issue.

Thank you for your fast response.

Regards,
Lienhart

[Kamal Mostafa (kamalmostafa)]

Proposed fix to upstream: https://github.com/boto/s3transfer/pull/88

[Christian Ehrhardt  (paelzer)]

This is still hanging on the GH review since ~a month.
@Kamal are you still driving this actively or just waiting on upstream in this case?

[Kamal Mostafa (kamalmostafa)]

@Christian, indeed I'm also just waiting on upstream. I've just added another comment to the github PR to ping the maintainer, kyleknap.

[Peter C Baughman (pbaughman)]

I think this open bug has the same root cause https://bugs.launchpad.net/ubuntu/+source/python-botocore/+bug/1711737

[Athos Ribeiro (athos-ribeiro)]

This has been fixed in https://github.com/boto/s3transfer/pull/102 and released upstream in s3transfer 0.3.7.

Note that Debian currently has version 0.5.0-1 in unstable. Therefore, once JJ is open for development and the syncs start, this should be fixed there.

I am also updating the affected series to include the new ones and closing the ones whose standard support no longer applies.

[Athos Ribeiro (athos-ribeiro)]

Finally, the patch in https://github.com/boto/s3transfer/pull/102 should be backported to the affected series

[Brian Murray (brian-murray)]

The Hirsute Hippo has reached End of Life, so this bug will not be fixed for that release.

[Lena Voytek (lvoytek)]

Are we still planning on backporting the fix to bionic, focal, and impish?

[Utkarsh Gupta (utkarsh)]

Bumping this.

[Brian Murray (brian-murray)]

Ubuntu 21.10 (Impish Indri) has reached end of life, so this bug will not be fixed for that specific release.

[Bryce Harrington (bryce)]

The three patches from the upstream bug, that would need backported:

https://github.com/boto/s3transfer/commit/d9eb4e611b859969c3e13e6dd9f43b8439424e25.patch
https://github.com/boto/s3transfer/commit/eecc4009f8191d384b40ef859b33960d7d80b731.patch
https://github.com/boto/s3transfer/commit/fe4f945b1eedf425e02ec368dd064bf26e4e6e5d.patch