A complete multipart upload request verification failed due to object-server unable to update container DB
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Medium
|
Unassigned | ||
Swift3 |
Invalid
|
Undecided
|
Unassigned |
Bug Description
This issue happens in some high latency backend network between object-server and container-server.
We saw client got 400 errors due to a complete multipart-upload request failed because there is one part isn't exist when Swift3 middleware try to get a list of all parts. And it caused by object-server unable to reach container-server in 0.5s (a connection timeout is 0.5s in default). So that's why swift3 unable to see the part in a container listing.
After 5 mins, object-updater try to get all async_pendings and sync to container DB and the client should able to see the part when doing container listing.
It causes random errors `One or more of the specified parts could not be found. The part might not have been uploaded, or the specified entity tag might not have matched the part's entity tag.` when clients try to upload object with multipart-upload. You won't see this in every multipart upload, but you'll see it sometimes.
I think following Swift 2.4.0 [1], the container listing should be unnecessary. We should be able to generate the manifest to PUT based off of our segment naming conventions and the client-provided etags. I need to check what happens if the client uploads more parts than are referenced in the finalization request, though.
[1] https:/ /github. com/openstack/ swift/commit/ bb71657