Concurrent uploads to same object

[rehanift]

2 files, one large (10M) and one small (1k).

Start uploading larger object, then start uploading smaller object. Both objects are writing to the same object URL

Small object finishes uploading and responds with small object's MD5 in ETag. Describing the object reproduces again the small object's MD5 in ETag. Large object is still uploading

Large object finishes uploading and responds with large objects large object's MD5 in ETag. Describing the object now responds with small object's MD5 in ETag.

I would expect the large object to respond with an error and/or with the small object's MD5 in the ETag header.

I reproduced this on Essex.

[Mike Barton (redbo)]

Yep, uploads are given a timestamp when they start, and whichever has the newest timestamp wins.

It should be possible to make the larger upload respond with an error when it's thrown away.

[rehanift (rehan-iftikhar)]

Is this a confirmed bug then? Or should this become a feature request?

[Tong Li (litong01)]

this is actually how Swift behaves. If the versioning is turned on, these actions make two versions. It should not be a bug. If will be
a bug if the two uploads actually mixed up the content into one version. I suggest to close this one without action.

[Mike Barton (redbo)]

I think returning an error if your upload is thrown away at the end is sort of reasonable.

[Kun Huang (academicgareth)]

@Mike
How do I know my upload is thrown away? Is there maybe a *.lock?

[Christian Schwede (cschwede)]

Due to the eventual consistency one can't ensure to return the smallers object etag when finishing the larger upload.

Let's assume the smaller upload used some handoff nodes (for whatever reason), and the small upload finished.
The larger upload is still in progress, and finally finishes too. Now the server checks if the stored etag is identical to the uploaded data - and returns the etag of the larger object, because that is the object stored on the primary nodes.

Now the client requests the etag again. In the meantime the replicators moved the smaller object from the handoffs to the primary nodes, and now you're getting the etag of the smaller object.

From my point of view this is just how Swift works - thus I'm closing the bug, because due to the eventual consistency there is no way to be sure which object is the newest in the cluster.