Comment 1 for bug 1878056

Reviewed: https://review.opendev.org/726570
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=dee98a74d43771d48a58d62647a0628ef7d1cf76
Submitter: Zuul
Branch: master

commit dee98a74d43771d48a58d62647a0628ef7d1cf76
Author: Tim Burke <email address hidden>
Date: Sat May 9 23:16:04 2020 -0700

    updater: Shuffle suffixes so we don't keep hitting the same failures

    When tuning your updater, you often want to try a new config, see how it
    changes your metrics, then adjust concurrency up or down depending on
    how your container layer is responding.

    If your containers haven't been doing well, though, and you've got a
    giant backlog of async pendings to work through, updater restarts to
    change concurrency previously posed a problem: the updater would walk
    the suffix directories in the same order every start-up. So, if you
    found a config that was making decent progress for a while but still had
    *some* failures, and you wanted to try tweaking settings to see if you
    could *reduce* those failures -- you'd likely start getting *all*
    failures as it went to retry the failed ones first and all at once. If
    you continued trying to tweak configs to get your failures to a
    reasonable rate, you'd almost certainly over-correct for these handful
    of overwhelmed DBs and not the overall cluster.

    Now, shuffle the suffixes before we walk them.

    Change-Id: I3ef34119f0cb563ab405a6517335a24dbaf2b4c3
    Closes-Bug: #1878056