[Horizon] Performance leak in "metering" dashboard

[Ilya Tyaptin]

Some types of requests to ceilometer in metering dashboard is not optimized. So we have a performance leak in environments with huge amount of ceilometer data.

Main bottlenecks which I see:
* Using a meter-list for getting a "unit" for metric. Meter list get all available metric on environment, it's may be a thousands of records. It's better to use sample-list for this metric with "limit" 1. It gets a 1 sample for metric.
* 7 day period by default for collecting statistics. On huge amount of data it's may freeze the horizon and ceilometer api, it's better to get one day statistics by default.

Upstream bug: https://bugs.launchpad.net/horizon/+bug/1444456

[Ivan Berezovskiy (iberezovskiy)]

https://review.fuel-infra.org/#/c/5763/

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/5991

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/5991

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/6002

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Review: https://review.fuel-infra.org/6002

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/6110

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Review: https://review.fuel-infra.org/6110

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ilya Tyaptin <email address hidden> on branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/5991

[Fuel Devops McRobotson (fuel-devops-robot)]

Change restored by Ilya Tyaptin <email address hidden> on branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/5991

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/6112

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/6112

[OSCI Robot (oscirobot)]

Reviewed: https://review.fuel-infra.org/6112
Committed: https://review.fuel-infra.org/gitweb?p=openstack/horizon.git;a=commitdiff;h=1a31713ac5c522b407c832a0677dd29e0461e795
Submitter: mos-infra-ci
Branch: openstack-ci/fuel-6.1/2014.2

commit 1a31713ac5c522b407c832a0677dd29e0461e795
Author: Ilya Tyaptin <email address hidden>

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: Idb925cc411fa69dfdbd15c3a8c64fcb060cd601a
Related-bug: #1444553

[OSCI Robot (oscirobot)]

Reviewed: https://review.fuel-infra.org/6110
Committed: https://review.fuel-infra.org/gitweb?p=openstack/horizon.git;a=commitdiff;h=cc126c02724469c6766646d371081b10104f1a11
Submitter: mos-infra-ci
Branch: openstack-ci/fuel-5.1.2/2014.1.1

commit cc126c02724469c6766646d371081b10104f1a11
Author: Ilya Tyaptin <email address hidden>

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: Ie947bbac83dca6178c023af4aa9d000bc4d42b51
Related-bug: #1444553

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/5991
Submitter: Timur Sufiev <email address hidden>
Branch: openstack-ci/fuel-6.1/2014.2

Commit: f91b364f38ea6a33582912cb693690d391844ded
Author: Ilya Tyaptin <email address hidden>
Date: Wed Apr 22 16:03:34 2015

Optimize getting unit in "metering" dashboard

Currently we using a meter-list for getting a "unit" for metric.
Meter list get all available metric on environment,
it's may be a thousands of records.
It's better to use sample-list for this metric with "limit" 1,
for geting "unit" from a 1 sample with queried metric.

Change-Id: I4bf0ab7ebcbcae1ddc90d8ee01d5f5a84ed4fbba
Related-bug: #1444553

[Pavel Vaylov (pvaylov)]

Hey guys,

How this bug can be fixed for existin 6.0 installation ?

[Vitaly Gusev (vgusev)]

Verified on ISO:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2-6.1"
  api: "1.0"
  build_number: "337"
  build_id: "2015-04-21_22-54-31"
  nailgun_sha: "3928c5aee6e7aabad37cf0665acc6790ef220141"
  python-fuelclient_sha: "08640730176591a3818f24e75b710f8c95846e6e"
  astute_sha: "bf1751a4fe0d912325e3b4af629126a59c1b2b51"
  fuel-library_sha: "af4eee78a4dd6e6606079f5515bac91c45b04114"
  fuel-ostf_sha: "df8db1f48f03b18126ce5ec65317a1eb83a5a95f"
  fuelmain_sha: "aad0a7ad97a5660f5d53fa830b168d36fa9694eb"

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/8157

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/8158

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Timur Sufiev <email address hidden> on branch: openstack-ci/fuel-7.0/2015.1.0
Review: https://review.fuel-infra.org/8157
Reason: Existing patch is incompatible with 7.0 branch

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Timur Sufiev <email address hidden> on branch: openstack-ci/fuel-5.1.2/2014.1.1
Review: https://review.fuel-infra.org/6002
Reason: Version 5.1.2 is not going to be released.

[Timur Sufiev (tsufiev-x)]

Here is the first one of 2 fixes to solve this issue in 7.0: https://review.fuel-infra.org/#/c/9422/ (merged).

[Dmitry Mescheryakov (dmitrymex)]

I did some manipulations with bug to restore state of 6.1.x series + target 7.0.x to 7.0 (it was targeted to 6.1 before).

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/9986

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Timur Sufiev <email address hidden> on branch: openstack-ci/fuel-7.0/2015.1.0
Review: https://review.fuel-infra.org/8158
Reason: Using https://review.fuel-infra.org/#/c/9986/ instead

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9986
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: d0eaef1fa9b8949559121886f49d0dcad3b95c52
Author: Ilya Tyaptin <email address hidden>
Date: Thu Jul 30 15:16:51 2015

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: I6f2ff071bb28ed9334c727072c16963ea2505de1
Closes-Bug: #1444553

[Vitaly Gusev (vgusev)]

Verified on ISO:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "165"
  build_id: "2015-08-12_17-24-26"
  nailgun_sha: "68642a8207d6f12543f244bab0c130e2510536ee"
  python-fuelclient_sha: "26fc025e0fc5791b62e5ed8561a6016bf8a406bc"
  fuel-agent_sha: "57145b1d8804389304cd04322ba0fb3dc9d30327"
  fuel-nailgun-agent_sha: "e01693992d7a0304d926b922b43f3b747c35964c"
  astute_sha: "e1d3a435e5df5b40cbfb1a3acf80b4176d15a2dc"
  fuel-library_sha: "1176b634eeafb8465a88ff357fdcf40005fba610"
  fuel-ostf_sha: "58220583f10fa47f12291488ef77854809c68310"
  fuelmain_sha: "67e5214c0dc5d4ba6da4ae651cef9934800459a9"

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/13363

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-6.0-updates/2014.2
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/14141

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/13363
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: 199b4145423f7cb1d55a7deb8abe12cd375c8cf0
Author: Ilya Tyaptin <email address hidden>
Date: Thu Nov 26 14:23:45 2015

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: I6f2ff071bb28ed9334c727072c16963ea2505de1
Closes-Bug: #1444553

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/14141
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-6.0-updates/2014.2

Commit: 5710a766a91853c88b0a96eb2e070d601728d157
Author: Ilya Tyaptin <email address hidden>
Date: Tue Nov 24 11:10:06 2015

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: Ie947bbac83dca6178c023af4aa9d000bc4d42b51
Related-bug: #1444553

[Vitaly Gusev (vgusev)]

Verified on Ubuntu 6.0.
Packages:
python-django-horizon
Version:
1:2014.2-fuel6.0~mira20

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0/mitaka
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/18473

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18473
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: f580311b4fb3712bfc8d3254cc2abb5ba188fb80
Author: Ilya Tyaptin <email address hidden>
Date: Thu Mar 24 10:32:03 2016

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: I6f2ff071bb28ed9334c727072c16963ea2505de1
Closes-Bug: #1444553

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/newton
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33862

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 11.0/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33906

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/34682

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ihor Kalnytskyi <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/33906
Reason: 11.0/ocata is obsolete. We use mcp/ocata instead.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ivan Kolodyazhny <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34682
Reason: It was removed in upstream

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/33862
Submitter: Pkgs Jenkins <email address hidden>
Branch: mcp/newton

Commit: 72339dde34c914e550161583ee3a44fd0be1f75f
Author: Ilya Tyaptin <email address hidden>
Date: Mon May 8 12:23:12 2017

Add a 1 hour period to "metering" dashboard

Currently we use "7 day" period by default in stats tab.
On huge amount of data it's may freeze the horizon and ceilometer api.
In this CR the "1 hour" period have been added and set by default.
It helps to avoid api freeze in dashboard opening and improve user
experience for this dashboard.

Change-Id: I6f2ff071bb28ed9334c727072c16963ea2505de1
Closes-Bug: #1444553