size issue for multipart with s3 backend

[Damien RANNOU]

When nova-compute is uploading a snapshot of a Vm, it is streaming the content of the image in a HTTP Chunked mode.
In this mode, there is no "Content-Lenght" header that is provided for the image (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#directives) so image_size is considered as zero

In the S3 backend a test is made based on the size.
In our case, the image_size is empty, so the upload will be done in singlepart mode:
https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/s3.py#L632

of course, be in singlepart might work, but it's less performant

Expected result

As a Vm snapshot is by design big, we should always send an image in multipart

Actual result

The image is always sent in single part

[Jasper Ras (jrasper)]

We are also seeing this same behaviour. It is troublesome in the case of S3 because the max PUT size is 5GB [1]. With snapshots practically always being bigger than this it is quite a pressing issue actually.

"Q: How much data can I store in Amazon S3?

The total volume of data and number of objects you can store in Amazon S3 are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB. For objects larger than 100 MB, customers should consider using the multipart upload capability."

[1] https://aws.amazon.com/s3/faqs/#:~:text=Individual%20Amazon%20S3%20objects%20can,single%20PUT%20is%205%20GB.