EC: Client Disconnect leaves inaccessible data on disk
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Critical
|
clayg |
Bug Description
If a client disconnects from a regular content-length put without sending all of the data a non-durable datafile is left on disk [1]
Since we're not returning non-druable fragments the proxy will still 404 a subsequent GET - but the data on disk should not be there - that object is not valid.
I attached a python script to play around with the disconnect scenario - but you still need to `find /srv/node*
This is probably the root cause of lp bug #1469094 - I found it riffing on tests Kota's attached to patch 211338 [2]
I'll probably try to codify this into failing probetest tomorrow if the overnight crew doesn't beat me to it.
1. replication on disconnect, I verified, will not only *not* drop the .data file in to the hashpath - it'll even clean up the tmpfile!
2. https:/
tags: | added: ec |
Changed in swift: | |
status: | New → Confirmed |
Changed in swift: | |
importance: | Undecided → Critical |
Changed in swift: | |
milestone: | none → 2.5.0 |
status: | Fix Committed → Fix Released |
I like this test - it uses a real proxy and a real object server on a real socket to really disconnect - and then it looks at the entire state of the world on-disk to show whats wrong. It has a test for repl that passes. It re-phrases the same test for ec with little code duplication and it fails. it reads ok I think. it cleans up after itself.
They say most of the time writing the test is the hard part... although I'm not sure on this one. We'll see!