object-updater should remember redirects and proactively check whether an update should be pointing at a shard instead

[Tim Burke]

One of the troubles you get with a large container is that async pendings start piling up pointing at it because the container can't keep up. Once you shard it, the container server will issue redirects to appropriate shards, but it can still only send so many at a time.

Instead of making two requests for every async pending (one to get the redirect, one to update the shard instead), the updater should get the whole set of shards loaded into its head after the first redirect, and use that information to avoid further requests to the root.

[clayg (clay-gerrard)]

Is there any rule that says the container server can't return the shards list as part of the 301?

Should there be any limit on how many shards and how long the updater can cache these?

[Tim Burke (1-tim-z)]

We absolutely could include the shards in the 301 -- that's the way to do it. There probably should be a (likely configurable) limit; ten containers' worth of shard ranges might be a reasonable default? Any time limit can likely be pretty long -- as long as it's within a reclaim age, the shard will redirect if it needed to shrink or shard.