commit 6474be2318f472b9ae4f2d0fae1174d9a5abf113
Author: gordon chung <email address hidden>
Date: Fri Feb 12 11:28:58 2016 -0500
partition unprocessed measures across workers
this patch assigned each worker to work on a specific partition of
measures backlog. it uses the metric reporting worker to dynamically
set the size of each partition.
by default each processing worker will handle a specific block of 128
metrics. the reporting worker will periodically check backlog size
and broadcast to all processing workers whether to grab larger chunks
(if backlog is big) or smaller chunks (when backlog is small).
this minimises overlap between workers. it may result in no-op
workers if backlog is very small. overlap may occur as the workers
query backlog at different times and may possibly have different block
sizes. overlap will occur the most in ceph driver because there is no
way to grab metrics as a whole but it should still minimise the overlap
to an extent.
a minimum block size is set at 16 metrics and a maximum of 256.
in theory if 8 workers are set per service, it will grab roughly 128
to 2048 unique metrics at a time.
Reviewed: https:/ /review. openstack. org/279659 /git.openstack. org/cgit/ openstack/ gnocchi/ commit/ ?id=6474be2318f 472b9ae4f2d0fae 1174d9a5abf113
Committed: https:/
Submitter: Jenkins
Branch: master
commit 6474be2318f472b 9ae4f2d0fae1174 d9a5abf113
Author: gordon chung <email address hidden>
Date: Fri Feb 12 11:28:58 2016 -0500
partition unprocessed measures across workers
this patch assigned each worker to work on a specific partition of
measures backlog. it uses the metric reporting worker to dynamically
set the size of each partition.
by default each processing worker will handle a specific block of 128
metrics. the reporting worker will periodically check backlog size
and broadcast to all processing workers whether to grab larger chunks
(if backlog is big) or smaller chunks (when backlog is small).
this minimises overlap between workers. it may result in no-op
workers if backlog is very small. overlap may occur as the workers
query backlog at different times and may possibly have different block
sizes. overlap will occur the most in ceph driver because there is no
way to grab metrics as a whole but it should still minimise the overlap
to an extent.
a minimum block size is set at 16 metrics and a maximum of 256.
in theory if 8 workers are set per service, it will grab roughly 128
to 2048 unique metrics at a time.
Closes-Bug: #1543121 be85e343e38c073 8c2b33efb92
Change-Id: Iad7b6b9a56fefc