[ceilometer] Huge amount of meaningless samples/events

[Ilya Tyaptin]

Currently Ceilometer process and saves all notification which defined in meters.yaml. Many of them are duplicated as events or don't carry any billing or monitoring meaning. It's a volume.update.start, identity.authenticate and etc.
Huge amount of these samples bulk list commands output and create additional disk load.

So, we suggest to avoid storing meters which don't carry any billing or monitoring meaning by default, but keep availability to turn it on in the config files.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/14567

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/15539

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/15539
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: a3e8b6901f0cd16c0560b45dfa3ae271c360b291
Author: Ilya Tyaptin <email address hidden>
Date: Fri Dec 25 15:09:34 2015

Fix a issue with failing nova notifications

Currently nova could send a notifications without tenant_id or user_id.
This "instance" notification are not recorded because notification descriptions
strictly need this params.
In this patch set we tried to fix it.

Change-Id: I865e2f664a5f146708e233b3342616be5405ec0c
Partially-closes-bug: #1524682

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/14567
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: c8d778e58eee54cddae63c541075876379a5a388
Author: Ilya Tyaptin <email address hidden>
Date: Fri Dec 25 15:09:34 2015

Add exlude rules for meaningless meters

This change request contains a declarative pipeline config fixing.
We exclude a non-metering meters which are duplicated by events,
polling meters which are mixed with notification and misleading
and samples/events which don't carry any meaning in monitoring or
billing context but bulk the db.

Change-Id: Ia5c8371823cb8b8e32d2638ff022e8a63468df01
Closes-bug: #1524682

[Vitaly Gusev (vgusev)]

Verified on ISO:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "466"
  build_id: "466"
  fuel-nailgun_sha: "f81311bbd6fee2665e3f96dcac55f72889b2f38c"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "6823f1d4005a634b8436109ab741a2194e2d32e0"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "fe03d887361eb80232e9914eae5b8d54304df781"
  fuel-ostf_sha: "ab5fd151fc6c1aa0b35bc2023631b1f4836ecd61"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "fac143f4dfa75785758e72afbdc029693e94ff2b"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "727f7076f04cb0caccc9f305b149a2b5b5c2af3a"

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18556
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: af6a1f49ccdce8c6ab9efcf45c8490387d538f59
Author: Ilya Tyaptin <email address hidden>
Date: Thu Mar 31 09:33:44 2016

Fix a issue with failing nova notifications

Currently nova could send a notifications without tenant_id or user_id.
This "instance" notification are not recorded because notification descriptions
strictly need this params.
In this patch set we tried to fix it.

Change-Id: I865e2f664a5f146708e233b3342616be5405ec0c
Partially-closes-bug: #1524682

[Timur Nurlygayanov (tnurlygayanov)]

This bug affects MOS 9.0 as well and it is already in progress for 9.0:
https://review.fuel-infra.org/#/c/18553/

Please merge the fix, many OSTF tests failed because of this issue.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18553
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: ca57c68cb172bfd5476dd66df1200a48e10ccad6
Author: Ilya Tyaptin <email address hidden>
Date: Thu Mar 31 09:38:24 2016

Add exlude rules for meaningless meters

This change request contains a declarative pipeline config fixing.
We exclude a non-metering meters which are duplicated by events,
polling meters which are mixed with notification and misleading
and samples/events which don't carry any meaning in monitoring or
billing context but bulk the db.

Change-Id: Ia5c8371823cb8b8e32d2638ff022e8a63468df01
Closes-bug: #1524682

[Vitaly Gusev (vgusev)]

Verified on ISO:
cat /etc/fuel_build_id:
 224
cat /etc/fuel_build_number:
 224
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6339.noarch
 fuel-misc-9.0.0-1.mos8306.noarch
 fuel-mirror-9.0.0-1.mos131.noarch
 fuel-openstack-metadata-9.0.0-1.mos8657.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8657.noarch
 python-fuelclient-9.0.0-1.mos312.noarch
 fuel-9.0.0-1.mos6339.noarch
 fuel-nailgun-9.0.0-1.mos8657.noarch
 rubygem-astute-9.0.0-1.mos738.noarch
 fuel-library9.0-9.0.0-1.mos8306.noarch
 network-checker-9.0.0-1.mos72.x86_64
 fuel-agent-9.0.0-1.mos273.noarch
 fuel-ui-9.0.0-1.mos2663.noarch
 fuel-setup-9.0.0-1.mos6339.noarch
 nailgun-mcagents-9.0.0-1.mos738.noarch
 python-packetary-9.0.0-1.mos131.noarch
 fuelmenu-9.0.0-1.mos269.noarch
 fuel-bootstrap-cli-9.0.0-1.mos273.noarch
 fuel-migrate-9.0.0-1.mos8306.noarch
 shotgun-9.0.0-1.mos88.noarch
 fuel-notify-9.0.0-1.mos8306.noarch
 fuel-ostf-9.0.0-1.mos924.noarch
 fuel-utils-9.0.0-1.mos8306.noarch

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/newton
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33181

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/33181
Submitter: Pkgs Jenkins <email address hidden>
Branch: mcp/newton

Commit: f41fa5a3476c18849272ff236090ded644ed91dc
Author: Ilya Tyaptin <email address hidden>
Date: Tue Apr 18 12:19:58 2017

Fix a issue with failing nova notifications

Currently nova could send a notifications without tenant_id or user_id.
This "instance" notification are not recorded because notification descriptions
strictly need this params.
In this patch set we tried to fix it.

Change-Id: I865e2f664a5f146708e233b3342616be5405ec0c
Partially-closes-bug: #1524682

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/newton
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33377

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 11.0/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33745

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 11.0/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/33785

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/34476

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/ocata
Change author: Ilya Tyaptin <email address hidden>
Review: https://review.fuel-infra.org/34491

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ilya Tyaptin <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34491

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/33785
Reason: we don't use 11.0/ocata anymore - mcp/ocata is the correct branch name

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/33745
Reason: we don't use 11.0/ocata anymore - mcp/ocata is the correct branch name

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ilya Tyaptin <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34476

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ilya Tyaptin <email address hidden> on branch: mcp/newton
Review: https://review.fuel-infra.org/33377