Comment 5 for bug 1605301

An easy way to understand this behavior is to create a ring with min_part_hours of 300 then add some devices and rebalance, then add another device, and pretend_min_part_hours and rebalance.

... nothing moves

With min_part_hours > 255 nothing will ever get gathered if it requires a last_part_moves check, no matter how far you are from the epoch apparently.

This is probably bad, it's a confusing semantic. Unfortunately I've leveraged this in the past to minimize unneeded partition movement during a gradual capacity increase that gets interrupted by a failed device:

    #!/bin/bash
    rm forever.builder || true
    swift-ring-builder forever.builder create 8 3 1
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d0 10
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d1 10
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d2 10
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d3 10
    swift-ring-builder forever.builder rebalance
    swift-ring-builder forever.builder
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d4 1
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d5 1
    swift-ring-builder forever.builder add r1z1-127.0.0.1:6001/d6 1
    echo "*** no parts can get reassigned!"
    swift-ring-builder forever.builder rebalance
    swift-ring-builder forever.builder
    echo "*** ... time passes"
    swift-ring-builder forever.builder pretend_min_part_hours_passed
    echo "*** gradually parts move"
    swift-ring-builder forever.builder rebalance
    swift-ring-builder forever.builder
    swift-ring-builder forever.builder set_weight d4 10
    swift-ring-builder forever.builder set_weight d5 10
    swift-ring-builder forever.builder set_weight d6 10
    echo "*** now it's un-balanced"
    swift-ring-builder forever.builder rebalance
    swift-ring-builder forever.builder
    echo "*** ... time passes"
    swift-ring-builder forever.builder pretend_min_part_hours_passed
    echo "*** and a device *fails!*"
    swift-ring-builder forever.builder remove d2
    cp forever.builder fast.builder
    echo "*** here can we do a little trick to prevent unneeded movement"
    swift-ring-builder forever.builder set_min_part_hours 300
    echo "*** ... and failed deivce parts reassign no matter what!"
    swift-ring-builder forever.builder rebalance | grep -i Reassigned
    echo "*** compared to not doing the trick"
    swift-ring-builder forever.builder set_min_part_hours 254
    swift-ring-builder fast.builder rebalance | grep -i Reassigned

In this scenario it's not unreasonable I think to want to ask rebalance to reassign *only* the parts it *has* to for the failed device. With the plan of waiting for that replication cycle to finish before continuing the normal gradual capacity adjustment and rebalance cycle.

It's possible this use-case is just an example of https://xkcd.com/1172/ ;)

But I think there's a subtle mistake in the bug report. It says "if min_part_hours is larger that 0xff, rebalance will never work" - but that's *only* true for some types of gather parts - the common ones people deal with when "playing" with rings. But in the real world there's some type of gather parts that must avoid min_part_hours checks, and when they do your goals as an operator can come into conflict with other priorities in the ring, so the only escape hatch we have gets used.

I'd really like to get some additional opinions to help align my thinking based on input from other operators and developers familiar with the ring and thinking about how its going to continue to evolve.