Grafana graphs drop down to 0 sometimes

[Simon Pasquier]

See attached screenshot. Instead of 'fill(0)' we should use 'fill(previous)' in the InfluxDB queries.

[Swann Croiset (swann-w)]

an other solution would be to set the 'Now delay' option to 30s to ignore the last 30 seconds

[Simon Pasquier (simon-pasquier)]

@Swann using 'Now delay' may not fix it everywhere since some metrics are only collected 60s.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/279114

[OpenStack Infra (hudson-openstack)]

Change abandoned by Simon Pasquier (<email address hidden>) on branch: master
Review: https://review.openstack.org/279114
Reason: This change missed the point completely!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/279422

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/279422
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-influxdb-grafana/commit/?id=9c969c7e7df6042ddc1810bc4364ffa36016554b
Submitter: Jenkins
Branch: master

commit 9c969c7e7df6042ddc1810bc4364ffa36016554b
Author: Simon Pasquier <email address hidden>
Date: Fri Feb 12 09:34:16 2016 +0100

    Use fill(previous) instead of fill(0)

    This change uses fill(previous) instead of fill(0) to avoid drops in the
    graphs when the latest data isn't available in InfluxDB yet.

    Change-Id: I4e540a08997b420140cb1c99397321e4a0f82273
    Closes-Bug: #1526383

[Swann Croiset (swann-w)]

The fix has an undesirable side effect: a graph stays flat (because filled w/ previous value) when no data:

This is particularly troubling for example for HTTP POST response time: see screenshot

IMO, it should be reverted.

We may consider the "Now delay" option instead.

[Simon Pasquier (simon-pasquier)]

'Now delay' isn't perfect either because you always get an outdated version of the metrics.

My take is:
1. revert to use fill(0)
2. use fill(previous) wherever it makes sense like, eg for metrics that are collected at less frequent intervals.

[Swann Croiset (swann-w)]

make sense, you are right 'Now delay' is global across all graphs and it's not ideal solution.

I reopened the bug and we should probably target it to 1.0

[Simon Pasquier (simon-pasquier)]

+1