Clean up copy-increasing code
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Dmedia |
Fix Released
|
Critical
|
Jason Gerard DeRose |
Bug Description
This branch started to fix "Critical: fix non-convergent scenario":
https:/
I still don't have a complete solution to lp:1247530, but lots of important refactoring and improvement has been done that should be landed on its own.
In general, the copy increasing code has been broken down into simpler pieces that are easier to unit test, lots of new unit tests have been added, and the copy-increasing behavior is now better tuned so that Dmedia converges much more quickly (in particular by prioritizing IO usage based on a new IO cost accounting model).
This branch also adds preemptive copy-increasing in order to provide better availability and so that a subset of files will typically be maintained at 4 copies (space permitting), which means there is typically some shuffling headroom. With this change, it's now far less likely that a library would get stuck in the non-convergent scenario discussed in lp:1247530.
But this preemptive copy-increasing doesn't provide a strong, bounded guarantee that lp:1247530 will never happen, so we're leaving the later open till we have a specific fix.
Related branches
- David Jordan: Approve
-
Diff: 1868 lines (+1306/-230)8 files modifieddmedia/core.py (+220/-55)
dmedia/local.py (+9/-9)
dmedia/metastore.py (+219/-53)
dmedia/tests/test_core.py (+144/-0)
dmedia/tests/test_local.py (+37/-0)
dmedia/tests/test_metastore.py (+490/-112)
dmedia/tests/test_views.py (+149/-0)
dmedia/views.py (+38/-1)
Changed in dmedia: | |
status: | In Progress → Fix Committed |
Changed in dmedia: | |
status: | Fix Committed → Fix Released |