Arithmetic transformer not working on multiple meters

[Avi Weit]

I am using ceilometer (devstack master on Ubuntu 14.04.3) with arithmetic transformer to combine two meters. I get "Unable to perform calculation" error (from ceilometer-notification service).

When using the attached pipeline.yaml with a disk_read_write_sink arithmetic transformer, I get:

"Unable to perform calculation, not all of {_disk_write_bytes_ESC, _disk_read_bytes_ESC} are present".

disk.read.bytes and disk.write.bytes properly polled.

Arithmetic transformer, with multiple meters, properly worked three months ago.

[gordon chung (chungg)]

for reference, do you know the two data points you are expecting to execute against? can you paste the samples here.

[gordon chung (chungg)]

@Avi, can you also paste your ceilometer.conf file (removing any passwords/ips)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/225144

[Avi Weit (weit)]

Attaching ceilometer.conf

[Avi Weit (weit)]

@Gordon,

Please find below a few samples for the two metrics for which arithmetic transformer depends on.

stack@~$ ceilometer sample-list -m disk.read.bytes -q 'resource_id=c8118316-b30d-4915-8100-aa99f97bdd33' --limit 3
+--------------------------------------+-----------------+------------+-----------+------+----------------------------+
| Resource ID | Name | Type | Volume | Unit | Timestamp |
+--------------------------------------+-----------------+------------+-----------+------+----------------------------+
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.read.bytes | cumulative | 1878346.0 | B | 2015-09-18T13:56:28.883705 |
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.read.bytes | cumulative | 1878346.0 | B | 2015-09-18T13:55:28.872261 |
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.read.bytes | cumulative | 1878346.0 | B | 2015-09-18T13:54:28.909757 |
+--------------------------------------+-----------------+------------+-----------+------+----------------------------+

stack@~$ ceilometer sample-list -m disk.write.bytes -q 'resource_id=c8118316-b30d-4915-8100-aa99f97bdd33' --limit 3
+--------------------------------------+------------------+------------+-----------+------+----------------------------+
| Resource ID | Name | Type | Volume | Unit | Timestamp |
+--------------------------------------+------------------+------------+-----------+------+----------------------------+
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.write.bytes | cumulative | 9458688.0 | B | 2015-09-18T13:56:28.924036 |
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.write.bytes | cumulative | 9458688.0 | B | 2015-09-18T13:55:28.916036 |
| c8118316-b30d-4915-8100-aa99f97bdd33 | disk.write.bytes | cumulative | 9458688.0 | B | 2015-09-18T13:54:28.944624 |
+--------------------------------------+------------------+------------+-----------+------+----------------------------+

[gordon chung (chungg)]

@Avi

thanks very much. i' believe i've solved it. if you'd like to verify, please try with above patch.

[Avi Weit (weit)]

@Gordon,

Arithmetic transformer seems to properly work now, when using your patch. I tested it with the attached pipeline on my devstack system and can see that samples of disk.read.write.bytes get properly produced.

Thanks.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/225144
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=1a18d3acaeab6754055805571938faa546f2b214
Submitter: Jenkins
Branch: master

commit 1a18d3acaeab6754055805571938faa546f2b214
Author: gordon chung <email address hidden>
Date: Fri Sep 18 09:29:40 2015 -0400

    remove batch processing requirement from arithmetic transformer

    arithmetic transformers relies on batch processing of samples. this
    is invalid assumption as samples are discrete datapoints and the
    transformer should not rely on polling logic to group datapoints
    especially now that we don't process data at polling agents.

    Change-Id: Id433bdef5ca3a59de49fc66e9805a37b4cd19781
    Closes-Bug: #1496701