partitioning logic runs too often

Bug #1554738 reported by gordon chung
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Gnocchi
Fix Released
Low
Trevor McCasland

Bug Description

gnocchi uses the reporting worker to partition tasks across each of the processing workers. by default reporter runs every 10s and processor worker every 5s. unfortunately, in any environment, small or large, the processor takes much longer than 10s to process it's data. because of that, the repartition messages that the reporter sends up queues up so the processor ends up running through a list of stale repartition messages before it can grab a new set of metrics.

we should either set reporter to report less frequently OR get it broadcast less frequently. i'm inclined to put a min period of 1min that the reporter can run at.

Julien Danjou (jdanjou)
Changed in gnocchi:
status: New → Triaged
importance: Undecided → Low
Changed in gnocchi:
assignee: nobody → Trevor McCasland (twm2016)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to gnocchi (master)

Fix proposed to branch: master
Review: https://review.openstack.org/290542

Changed in gnocchi:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to gnocchi (master)

Reviewed: https://review.openstack.org/290542
Committed: https://git.openstack.org/cgit/openstack/gnocchi/commit/?id=d87048d34693bc3ee45b8fa152f133070f9c3f7f
Submitter: Jenkins
Branch: master

commit d87048d34693bc3ee45b8fa152f133070f9c3f7f
Author: twm2016 <email address hidden>
Date: Wed Mar 9 08:36:39 2016 -0600

    Increase partitioning logic period

    gnocchi uses the reporting worker to partition tasks across each of
    the processing workers. by default reporter runs every 10s and
    processor worker every 5s. unfortunately, in any environment, small
    or large, the processor takes much longer than 10s to process it's
    data. because of that, the repartition messages that the reporter
    sends up queues up so the processor ends up running through a list
    of stale repartition messages before it can grab a new set of
    metrics.

    This calls for an increase in the period of which the reporter runs.

    Change-Id: Ie7b114a1c8f55e7291057e8193172c4f188a2f61
    Closes-Bug: #1554738

Changed in gnocchi:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to gnocchi (stable/2.0)

Fix proposed to branch: stable/2.0
Review: https://review.openstack.org/291706

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to gnocchi (stable/2.0)

Reviewed: https://review.openstack.org/291706
Committed: https://git.openstack.org/cgit/openstack/gnocchi/commit/?id=b0f5df4fed38c40df5876a2bcdb5f7c673765481
Submitter: Jenkins
Branch: stable/2.0

commit b0f5df4fed38c40df5876a2bcdb5f7c673765481
Author: twm2016 <email address hidden>
Date: Wed Mar 9 08:36:39 2016 -0600

    Increase partitioning logic period

    gnocchi uses the reporting worker to partition tasks across each of
    the processing workers. by default reporter runs every 10s and
    processor worker every 5s. unfortunately, in any environment, small
    or large, the processor takes much longer than 10s to process it's
    data. because of that, the repartition messages that the reporter
    sends up queues up so the processor ends up running through a list
    of stale repartition messages before it can grab a new set of
    metrics.

    This calls for an increase in the period of which the reporter runs.

    Change-Id: Ie7b114a1c8f55e7291057e8193172c4f188a2f61
    Closes-Bug: #1554738
    (cherry picked from commit d87048d34693bc3ee45b8fa152f133070f9c3f7f)

Julien Danjou (jdanjou)
Changed in gnocchi:
milestone: none → 2.1.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.