Performance leak in "metering" dashboard

[Ilya Tyaptin]

Some types of requests to ceilometer in metering dashboard are not optimized. So we have a performance hit in environments with huge amount of ceilometer data.

Main bottlenecks which I see are the following:
* Using a meter-list for getting a "unit" for metric. Meter list gets all available metrics on environment, it may be a thousands of records. It's better to use sample-list for this metric with "limit" 1. It gets only single sample for metric.
* 7 day period by default for collecting statistics. On huge amount of data it may freeze the Horizon and Ceilometer API, it's better to get one day statistics by default.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/173894

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/173961

[Matthias Runge (mrunge)]

This issue should be fixed by: https://github.com/openstack/horizon/commit/eabbaaeb490f048763b960cfaf450f39fd861b54

I agree, one day of metering data may still be too much data. How about to add an option to return the last hour or last day instead of hard limiting to a single result?

[Ilya Tyaptin (ityaptin)]

Matthias, thanks! Add an option to return the last hour and make it is chosen by default is good idea.
But I'm afraid what your commit doesn't fix described issue, because session has "date_option.period=7" by default [1].
Please, correct me, if i misunderstood.

Also I think you said "hard limiting to a single result" about CR [1]. I have already answered in this CR, hardcoded limit=1 there is not for presented data, it needs only for finding 'unit' name.

 [1]
 [2] https://review.openstack.org/173894

[Ilya Tyaptin (ityaptin)]

[1] https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/admin/metering/templates/metering/stats.html#L117

[Matthias Runge (mrunge)]

Ilya, your proposed patch is called only, when using configure metering.

https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/admin/metering/templates/metering/stats.html#L117
will be used as preset, when configuring a different time span and that was taken from previous commits.

https://github.com/openstack/horizon/commit/eabbaaeb490f048763b960cfaf450f39fd861b54
is called or used as a fallback, when there was no preset, which is stored in users session.

Thanks for updating the other review, you're right there.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/224853

[Matthias Runge (mrunge)]

I'm concerned about ceilometer as data source. It is not suitable for this kind of interaction, due to a architectural decision. We should switch to use gnocchi asap; Ceilometer accumulates data on the fly, where gnocchi uses pre-processing, which provices a HUGE speedup.

[gordon chung (chungg)]

+ to gnocchi. i believe the decision at Tokyo summit was to stop showing data from legacy ceilometer data. this tab should really be disabled by default and have a warning to tell users to use gnocchi.

regarding discussion about integrating gnocchi, we didn't go much further in how/when to do it. right now Gnocchi provides graphana integration for its visualisation. does this help with horizon integration?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/272644

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ilya Tyaptin (<email address hidden>) on branch: master
Review: https://review.openstack.org/173961

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ilya Tyaptin (<email address hidden>) on branch: master
Review: https://review.openstack.org/224853

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rob Cresswell (<email address hidden>) on branch: master
Review: https://review.openstack.org/272644
Reason: This was handled in another patch

[Akihiro Motoki (amotoki)]

The metering panel has been dropped in horizon in several releases ago.