when min_part_hours is zero composite rebalance may move same partition in multiple cobuilders
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| OpenStack Object Storage (swift) |
High
|
Alistair Coles |
Bug Description
CompositeRingBu
The proposd patch will include a test that illustrates the bug.
[1] It might be argued that zero min_part_hours suggests it is ok for multiple co-builders to move the same partition - "it is fine to move this part again in a subsequent rebalance with no delay". However, IMO that is not the design goal of composite rebalance, where each component rebalance is not a "subsequent rebalance". Users may have other means of preventing subsequent rebalance of single or composite rings occurring too often, which makes the use of zero min_part_hours simply a convenience. Composite rebalance must enforce single partition movement regardless of min_part_hours.
Alistair Coles (alistair-coles) wrote : | #1 |
Changed in swift: | |
status: | New → In Progress |
assignee: | nobody → Alistair Coles (alistair-coles) |
Changed in swift: | |
importance: | Undecided → High |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 348bd83b7eb638d
Author: Alistair Coles <email address hidden>
Date: Wed Aug 30 15:29:14 2017 +0100
Respect co-builder partition moves when min_part_hours is zero
Repeated calls to each co-builder's _update_
unnecessary and have the unfortunate side effect of resetting the
_last_
min_part_hours this results in it not preventing its co-builders from
moving parts that it has already moved.
This patch changes the CompositeRingBu
builder _update_
component builder. CooperativeRing
their _update_
_last_
of the composite rebalance.
The initialisation of the RingBuilder _last_part_moves array is moved
to the RingBuilder __init__ method, so that calls to
_update_
never been called on that builder. Otherwise, during a composite
rebalance, a component that has not previously been rebalanced will
not have its _last_part_
result may report erroneous min_part_
rebalance.
Related-Change: I1b30cb3d776be4
Closes-Bug: #1714274
Change-Id: Ib165cf974c865d
Changed in swift: | |
status: | In Progress → Fix Released |
Fix proposed to branch: feature/deep
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: feature/deep
commit 77a8a4455d4a2bd
Author: Tim Burke <email address hidden>
Date: Tue Oct 3 00:11:07 2017 +0000
Let clients request heartbeats during SLO PUTs
An SLO PUT requires that we HEAD every referenced object; as a result, it
can be a very time-intensive operation. This makes it difficult as a
client to differentiate between a proxy-server that's still doing work and
one that's crashed but left the socket open.
Now, clients can opt-in to receiving heartbeats during long-running PUTs
by including the query parameter
With heartbeating turned on, the proxy will start its response immediately
with 202 Accepted then send a single whitespace character periodically
until the request completes. At that point, a final summary chunk will be
sent which includes a "Response Status" key indicating success or failure
and (if successful) an "Etag" key indicating the Etag of the resulting SLO.
This mechanism is very similar to the way bulk extractions and deletions
work, and even the way SLO behaves for ?multipart-
Note that this is opt-in: this prevents us from sending the 202 response
to existing clients that may mis-interpret it as an immediate indication
of success.
Co-Authored-By: Alistair Coles <email address hidden>
Related-Bug: 1718811
Change-Id: I65cee5f629c873
commit 92705bb36b4a771
Author: David Rabel <email address hidden>
Date: Thu Nov 2 12:38:41 2017 +0100
Fix indent in overview_
Change-Id: I7f070956d8b996
commit feee3998408e5ed
Author: Clay Gerrard <email address hidden>
Date: Fri Sep 1 14:15:45 2017 -0700
Use check_drive consistently
We added check_drive to the account/container servers to unify how all
the storage wsgi servers treat device dirs/mounts. Thus pushes that
unification down into the consistency engine.
Drive-by:
* use FakeLogger less
* clean up some repeititon in probe utility for device re-"mounting"
Related-
Change-Id: I941ffbc568ebfa
commit 29e9ae1cc5b74dc
Author: Tim Burke <email address hidden>
Date: Thu Oct 19 18:53:04 2017 +0000
Make xml responses less insane
Looking at bulk extractions:
$ tar c *.py | curl http://
-H accept:
<?xml version="1.0" encoding="UTF-8"?>
<delete>
<errors>
</errors>
</delete>
Or SLO upload failures:
$ curl http://
tags: | added: in-feature-deep |
Fix proposed to branch: feature/s3api
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: feature/s3api
commit 77a8a4455d4a2bd
Author: Tim Burke <email address hidden>
Date: Tue Oct 3 00:11:07 2017 +0000
Let clients request heartbeats during SLO PUTs
An SLO PUT requires that we HEAD every referenced object; as a result, it
can be a very time-intensive operation. This makes it difficult as a
client to differentiate between a proxy-server that's still doing work and
one that's crashed but left the socket open.
Now, clients can opt-in to receiving heartbeats during long-running PUTs
by including the query parameter
With heartbeating turned on, the proxy will start its response immediately
with 202 Accepted then send a single whitespace character periodically
until the request completes. At that point, a final summary chunk will be
sent which includes a "Response Status" key indicating success or failure
and (if successful) an "Etag" key indicating the Etag of the resulting SLO.
This mechanism is very similar to the way bulk extractions and deletions
work, and even the way SLO behaves for ?multipart-
Note that this is opt-in: this prevents us from sending the 202 response
to existing clients that may mis-interpret it as an immediate indication
of success.
Co-Authored-By: Alistair Coles <email address hidden>
Related-Bug: 1718811
Change-Id: I65cee5f629c873
commit 92705bb36b4a771
Author: David Rabel <email address hidden>
Date: Thu Nov 2 12:38:41 2017 +0100
Fix indent in overview_
Change-Id: I7f070956d8b996
commit feee3998408e5ed
Author: Clay Gerrard <email address hidden>
Date: Fri Sep 1 14:15:45 2017 -0700
Use check_drive consistently
We added check_drive to the account/container servers to unify how all
the storage wsgi servers treat device dirs/mounts. Thus pushes that
unification down into the consistency engine.
Drive-by:
* use FakeLogger less
* clean up some repeititon in probe utility for device re-"mounting"
Related-
Change-Id: I941ffbc568ebfa
commit 29e9ae1cc5b74dc
Author: Tim Burke <email address hidden>
Date: Thu Oct 19 18:53:04 2017 +0000
Make xml responses less insane
Looking at bulk extractions:
$ tar c *.py | curl http://
-H accept:
<?xml version="1.0" encoding="UTF-8"?>
<delete>
<errors>
</errors>
</delete>
Or SLO upload failures:
$ curl http://
tags: | added: in-feature-s3api |
This issue was fixed in the openstack/swift 2.16.0 release.
Fix proposed here https:/ /review. openstack. org/#/c/ 499634/ 1