Swift3 is not write affinity aware, that causes multipart upload failed in `404 (NoSuchKey)`

[Charles Hsu]

Here is the flow I tested.

1. Enable write affinity
2. Stop all object-replicator on each node
3. Do a multipart upload

$ s3cmd put test.zip s3://test/test10.zip
upload: 'test.zip' -> 's3://test/test10.zip' [part 1 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 7.23 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 2 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 6.49 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 3 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 9.58 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 4 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 7.55 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 5 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 6.76 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 6 of 7, 5MB] [1 of 1]
 5242880 of 5242880 100% in 0s 7.84 MB/s done
upload: 'test.zip' -> 's3://test/test10.zip' [part 7 of 7, 2048kB] [1 of 1]
 2097152 of 2097152 100% in 0s 4.72 MB/s done
ERROR: S3 error: 404 (NoSuchKey): The specified key does not exist.

Here is detail logs from s3cmd and swift.

[Charles Hsu (charles0126)]

Patch: https://review.openstack.org/#/c/368264/2

[Charles Hsu (charles0126)]

Sorry! Here is the logs(https://gist.github.com/charz/4dbdcf188202ac121992526564e8aedd)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/499805
Committed: https://git.openstack.org/cgit/openstack/swift3/commit/?id=7524c3ace86d73082b6b3a75311c6d8d393b34fb
Submitter: Jenkins
Branch: master

commit 7524c3ace86d73082b6b3a75311c6d8d393b34fb
Author: Tim Burke <email address hidden>
Date: Thu Aug 31 20:47:17 2017 +0000

    Tolerate 404s when DELETEing multipart upload marker

    As long as we got a response when we HEADed the thing, that's good
    enough.

    Closes-Bug: 1638512
    Change-Id: I3d7a6ca50cf1a6c9dce08b9eb069eca2d4112922