Comment 3 for bug 1839355
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Fix merged to swift (master) | #3 |
© 2004
Canonical Ltd.
•
Terms of use
•
Data privacy
•
Contact Launchpad Support
•
Blog
•
Careers
•
System status
•
0e1f616
(Get the code!)
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit e9cd9f74a5264f3
Author: Matthew Oliver <email address hidden>
Date: Mon Aug 12 16:16:17 2019 +1000
sharder: Keep cleaving on empty shard ranges
When a container is being cleaved there is a possiblity that we're
dealing with an empty or near empty container created on a handoff node.
These containers may have a valid list of shard ranges, so would need
to cleave to the new shards.
Currently, when using a `cleave_batch_size` that is smaller then the
number of shard ranges on the cleaving container, these containers will
have to take a few shard passes to shard, even though there maybe
nothing in them.
This is worse if a really large container is sharding, and due to being
slow, error limitted a node causing a new container on a handoff
location. This empty container would have a large number of shard ranges
and could take a _very_ long time to shard away, slowing the process
down.
This patch eliminates the issue by detecting when no objects are
returned for a shard range. The `_cleave_
returns 3 possible results:
- CLEAVE_SUCCESS
- CLEAVE_FAILED
- CLEAVE_EMPTY
They all are pretty self explanitory. When `CLEAVE_EMPTY` is returned
the code will:
- Log
- Not replicate the empty temp shard container sitting in a
handoff location
- Not count the shard range in the `cleave_batch_size` count
- Update the cleaving context so sharding can move forward
If there already is a shard range DB existing on a handoff node to use
then the sharder wont skip it, even if there are no objects, it'll
replicate it and treat it as normal, including using a `cleave_batch_size`
slot.
Change-Id: Id338f6c3187f93
Closes-Bug: #1839355