Successful PUT object might be missing container update on EC
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Critical
|
Kota Tsuyuzaki |
Bug Description
At PUT object request, proxy server makes backend headers (e.g. X-Container-
the container-server they should update. In addition, the backend headers are created as many as the number of container replicas.
(i.e. 3 replica in container ring, 3 backend headers will be created)
On EC case, Swift fans out fragment archives to backend object-servers. Basically the number of fragment archives will be more than the container replica number and proxy-server assumes a request as success when quorum number of object-server succeeded to store. That would cause to make an object which is stored but not container updated.
For example, assuming k=10, m=4, container replica=3 case:
Assuming, proxy-server attempts to make 14 backend streams but unfortunately first 3 nodes returns 507 (disk failure) and then
the Swift doesn't have any other disks. In the case, proxy keeps 11 backend streams to store and current Swift assumes it as sufficient because it is more than or equals quorum (right now k+1 is sufficient i.e. 11 backend streams are enough to store)
However, in the case, the 11 streams doesn't have the container update header so that the request will succeed but container will be never updated.
Affected version is Erasure Code stuff on Swift 2.3.0.
Changed in swift: | |
assignee: | nobody → Kota Tsuyuzaki (tsuyuzaki-kota) |
Changed in swift: | |
status: | New → In Progress |
Changed in swift: | |
importance: | Undecided → High |
Changed in swift: | |
importance: | High → Critical |
Changed in swift: | |
milestone: | none → 2.5.0 |
status: | Fix Committed → Fix Released |
Fix proposed at https:/ /review. openstack. org/#/c/ 186735/