watcher

group_by statement in metric queries does not work within ceilosca

Bug #1623983 reported by David TARDIVEL on 2016-09-15

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	watcher	Fix Released	High	David TARDIVEL	watcher newton-rc-final "nrcfinal"

Bug Description

Here are tests with ipdb:

ipdb> rr = self.ceilometer.statistics.list(meter_name=meter_name, q=query, period=period)

2016-09-14 13:33:15.086 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mREQ: curl -g -i -X GET http://10.50.0.103:8777/v2/meters/compute.node.cpu.percent/statistics?q.field=resource_id&q.field=timestamp&q.field=timestamp&q.op=eq&q.op=ge&q.op=le&q.type=&q.type=&q.type=&q.value=rdev-indeedsrv002_rdev-indeedsrv002&q.value=2016-09-14T11%3A28%3A54.996700&q.value=2016-09-14T13%3A28%3A54.996694&period=7200 -H "User-Agent: python-ceilometerclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}57a910bfd52d12421c80516878ab131f1fdb3391"^[[00m ^[[00;33mfrom (pid=10410) _http_log_request /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:337^[[00m
2016-09-14 13:33:15.233 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mRESP: [200] Content-Length: 640 Content-Type: application/json; charset=UTF-8 x-openstack-request-id: req-f2a168c5-ac02-407e-83fb-50d515312204 Server: Werkzeug/0.9.4 Python/2.7.6 Date: Wed, 14 Sep 2016 13:33:15 GMT
RESP BODY: [{"count": 30, "duration_start": "2016-09-14T11:28:54.996700", "min": 50.0, "duration_end": "2016-09-14T12:00:00", "max": 51.0, "sum": 1504.0, "period": 7200, "period_end": "2016-09-14T12:00:00", "duration": 1865.0033, "period_start": "2016-09-14T10:00:00", "avg": 50.13333333333333, "groupby": {"": ""}, "unit": null}, {"count": 91, "duration_start": "2016-09-14T12:00:00", "min": 50.0, "duration_end": "2016-09-14T13:28:54.996694", "max": 51.0, "sum": 4572.0, "period": 7200, "period_end": "2016-09-14T14:00:00", "duration": 5334.996694, "period_start": "2016-09-14T12:00:00", "avg": 50.24175824175825, "groupby": {"": ""}, "unit": null}]
^[[00m ^[[00;33mfrom (pid=10410) _http_log_response /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:366^[[00m

ipdb> rr2 = self.ceilometer.statistics.list(meter_name=meter_name, q=query, period=period, aggregates=[{'func': aggregate}], groupby=['resource_id'])

2016-09-14 13:33:19.086 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mREQ: curl -g -i -X GET http://10.50.0.103:8777/v2/meters/compute.node.cpu.percent/statistics?q.field=resource_id&q.field=timestamp&q.field=timestamp&q.op=eq&q.op=ge&q.op=le&q.type=&q.type=&q.type=&q.value=rdev-indeedsrv002_rdev-indeedsrv002&q.value=2016-09-14T11%3A28%3A54.996700&q.value=2016-09-14T13%3A28%3A54.996694&period=7200&groupby=resource_id&aggregate.func=avg -H "User-Agent: python-ceilometerclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}57a910bfd52d12421c80516878ab131f1fdb3391"^[[00m ^[[00;33mfrom (pid=10410) _http_log_request /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:337^[[00m
2016-09-14 13:33:19.249 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mRESP: [200] Content-Length: 329 Content-Type: application/json; charset=UTF-8 x-openstack-request-id: req-1ff46d99-7215-4905-b098-0d91b1b6668c Server: Werkzeug/0.9.4 Python/2.7.6 Date: Wed, 14 Sep 2016 13:33:19 GMT
RESP BODY: [{"duration_start": "2016-09-14T11:28:54.996700", "duration_end": "2016-09-14T12:00:00", "period": 7200, "duration": 1865.0033, "period_end": "2016-09-14T12:00:00", "aggregate": {"avg": 0.0}, "period_start": "2016-09-14T10:00:00", "avg": 0.0, "groupby": {"resource_id": "rdev-indeedsrv002_rdev-indeedsrv002"}, "unit": "percent"}]
^[[00m ^[[00;33mfrom (pid=10410) _http_log_response /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:366^[[00m

ipdb> rr2
[<Statistics {u'duration_start': u'2016-09-14T11:28:54.996700', u'duration_end': u'2016-09-14T12:00:00', u'period': 7200, u'aggregate': {u'avg': 0.0}, u'period_end': u'2016-09-14T12:00:00', u'duration': 1865.0033, u'period_start': u'2016-09-14T10:00:00', u'avg': 0.0, u'groupby': {u'resource_id': u'rdev-indeedsrv002_rdev-indeedsrv002'}, u'unit': u'percent'}>]
ipdb> rr = self.ceilometer.statistics.list(meter_name=meter_name, q=query, period=period)
2016-09-14 13:34:46.833 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mREQ: curl -g -i -X GET http://10.50.0.103:8777/v2/meters/compute.node.cpu.percent/statistics?q.field=resource_id&q.field=timestamp&q.field=timestamp&q.op=eq&q.op=ge&q.op=le&q.type=&q.type=&q.type=&q.value=rdev-indeedsrv002_rdev-indeedsrv002&q.value=2016-09-14T11%3A28%3A54.996700&q.value=2016-09-14T13%3A28%3A54.996694&period=7200 -H "User-Agent: python-ceilometerclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}57a910bfd52d12421c80516878ab131f1fdb3391"^[[00m ^[[00;33mfrom (pid=10410) _http_log_request /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:337^[[00m
2016-09-14 13:34:46.976 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mRESP: [200] Content-Length: 640 Content-Type: application/json; charset=UTF-8 x-openstack-request-id: req-4c1147e5-e20a-40c9-ad8d-1cc1830752fd Server: Werkzeug/0.9.4 Python/2.7.6 Date: Wed, 14 Sep 2016 13:34:46 GMT
RESP BODY: [{"count": 30, "duration_start": "2016-09-14T11:28:54.996700", "min": 50.0, "duration_end": "2016-09-14T12:00:00", "max": 51.0, "sum": 1504.0, "period": 7200, "period_end": "2016-09-14T12:00:00", "duration": 1865.0033, "period_start": "2016-09-14T10:00:00", "avg": 50.13333333333333, "groupby": {"": ""}, "unit": null}, {"count": 93, "duration_start": "2016-09-14T12:00:00", "min": 50.0, "duration_end": "2016-09-14T13:28:54.996694", "max": 51.0, "sum": 4672.0, "period": 7200, "period_end": "2016-09-14T14:00:00", "duration": 5334.996694, "period_start": "2016-09-14T12:00:00", "avg": 50.23655913978495, "groupby": {"": ""}, "unit": null}]
^[[00m ^[[00;33mfrom (pid=10410) _http_log_response /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:366^[[00m

ipdb> rr2 = self.ceilometer.statistics.list(meter_name=meter_name, q=query, period=period, aggregates=[{'func': aggregate}])

2016-09-14 13:34:57.608 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mREQ: curl -g -i -X GET http://10.50.0.103:8777/v2/meters/compute.node.cpu.percent/statistics?q.field=resource_id&q.field=timestamp&q.field=timestamp&q.op=eq&q.op=ge&q.op=le&q.type=&q.type=&q.type=&q.value=rdev-indeedsrv002_rdev-indeedsrv002&q.value=2016-09-14T11%3A28%3A54.996700&q.value=2016-09-14T13%3A28%3A54.996694&period=7200&aggregate.func=avg -H "User-Agent: python-ceilometerclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}57a910bfd52d12421c80516878ab131f1fdb3391"^[[00m ^[[00;33mfrom (pid=10410) _http_log_request /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:337^[[00m
2016-09-14 13:34:57.774 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mRESP: [200] Content-Length: 614 Content-Type: application/json; charset=UTF-8 x-openstack-request-id: req-984719b0-1629-482c-9924-5766daea5e50 Server: Werkzeug/0.9.4 Python/2.7.6 Date: Wed, 14 Sep 2016 13:34:57 GMT
RESP BODY: [{"duration_start": "2016-09-14T11:28:54.996700", "duration_end": "2016-09-14T12:00:00", "period": 7200, "duration": 1865.0033, "period_end": "2016-09-14T12:00:00", "aggregate": {"avg": 50.13333333333333}, "period_start": "2016-09-14T10:00:00", "avg": 50.13333333333333, "groupby": {"": ""}, "unit": null}, {"duration_start": "2016-09-14T12:00:00", "duration_end": "2016-09-14T13:28:54.996694", "period": 7200, "duration": 5334.996694, "period_end": "2016-09-14T14:00:00", "aggregate": {"avg": 50.23655913978495}, "period_start": "2016-09-14T12:00:00", "avg": 50.23655913978495, "groupby": {"": ""}, "unit": null}]
^[[00m ^[[00;33mfrom (pid=10410) _http_log_response /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:366^[[00m

ipdb> pp rr
[<Statistics {u'count': 30, u'duration_start': u'2016-09-14T11:28:54.996700', u'min': 50.0, u'max': 51.0, u'duration_end': u'2016-09-14T12:00:00', u'period': 7200, u'groupby': {u'': u''}, u'period_end': u'2016-09-14T12:00:00', u'duration': 1865.0033, u'period_start': u'2016-09-14T10:00:00', u'avg': 50.13333333333333, u'sum': 1504.0, u'unit': None}>,
<Statistics {u'count': 93, u'duration_start': u'2016-09-14T12:00:00', u'min': 50.0, u'max': 51.0, u'duration_end': u'2016-09-14T13:28:54.996694', u'period': 7200, u'groupby': {u'': u''}, u'period_end': u'2016-09-14T14:00:00', u'duration': 5334.996694, u'period_start': u'2016-09-14T12:00:00', u'avg': 50.23655913978495, u'sum': 4672.0, u'unit': None}>]

ipdb> pp rr2
[<Statistics {u'duration_start': u'2016-09-14T11:28:54.996700', u'duration_end': u'2016-09-14T12:00:00', u'period': 7200, u'aggregate': {u'avg': 50.13333333333333}, u'period_end': u'2016-09-14T12:00:00', u'duration': 1865.0033, u'period_start': u'2016-09-14T10:00:00', u'avg': 50.13333333333333, u'groupby': {u'': u''}, u'unit': None}>,
<Statistics {u'duration_start': u'2016-09-14T12:00:00', u'duration_end': u'2016-09-14T13:28:54.996694', u'period': 7200, u'aggregate': {u'avg': 50.23655913978495}, u'period_end': u'2016-09-14T14:00:00', u'duration': 5334.996694, u'period_start': u'2016-09-14T12:00:00', u'avg': 50.23655913978495, u'groupby': {u'': u''}, u'unit': None}>]
ipdb> query
[{'field': 'resource_id', 'value': u'rdev-indeedsrv002_rdev-indeedsrv002', 'op': 'eq'}, {'field': 'timestamp', 'value': '2016-09-14T11:28:54.996700', 'op': 'ge'}, {'field': 'timestamp', 'value': '2016-09-14T13:28:54.996694', 'op': 'le'}]
ipdb> meter_name
'compute.node.cpu.percent'

ipdb> rr3 = self.ceilometer.statistics.list(meter_name=meter_name, q=query, period=period, aggregates=[{'func': aggregate}], groupby=['resource_id'])

2016-09-14 13:37:52.042 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mREQ: curl -g -i -X GET http://10.50.0.103:8777/v2/meters/compute.node.cpu.percent/statistics?q.field=resource_id&q.field=timestamp&q.field=timestamp&q.op=eq&q.op=ge&q.op=le&q.type=&q.type=&q.type=&q.value=rdev-indeedsrv002_rdev-indeedsrv002&q.value=2016-09-14T11%3A28%3A54.996700&q.value=2016-09-14T13%3A28%3A54.996694&period=7200&groupby=resource_id&aggregate.func=avg -H "User-Agent: python-ceilometerclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}57a910bfd52d12421c80516878ab131f1fdb3391"^[[00m ^[[00;33mfrom (pid=10410) _http_log_request /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:337^[[00m
2016-09-14 13:37:52.268 DEBUG keystoneauth.session [^[[00;36m-] ^[[01;35mRESP: [200] Content-Length: 329 Content-Type: application/json; charset=UTF-8 x-openstack-request-id: req-8a997c9d-8b1a-488a-aa5b-cab754b41007 Server: Werkzeug/0.9.4 Python/2.7.6 Date: Wed, 14 Sep 2016 13:37:52 GMT
RESP BODY: [{"duration_start": "2016-09-14T11:28:54.996700", "duration_end": "2016-09-14T12:00:00", "period": 7200, "duration": 1865.0033, "period_end": "2016-09-14T12:00:00", "aggregate": {"avg": 0.0}, "period_start": "2016-09-14T10:00:00", "avg": 0.0, "groupby": {"resource_id": "rdev-indeedsrv002_rdev-indeedsrv002"}, "unit": "percent"}]
^[[00m ^[[00;33mfrom (pid=10410) _http_log_response /opt/.virtualenvs/watcherdev/local/lib/python2.7/site-packages/keystoneauth1/session.py:366^[[00m

ipdb> pp rr3
[<Statistics {u'duration_start': u'2016-09-14T11:28:54.996700', u'duration_end': u'2016-09-14T12:00:00', u'period': 7200, u'aggregate': {u'avg': 0.0}, u'period_end': u'2016-09-14T12:00:00', u'duration': 1865.0033, u'period_start': u'2016-09-14T10:00:00', u'avg': 0.0, u'groupby': {u'resource_id': u'rdev-indeedsrv002_rdev-indeedsrv002'}, u'unit': u'percent'}>]
ipdb>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-15: Fix proposed to watcher (master)

Fix proposed to branch: master
Review: https://review.openstack.org/370938

Changed in watcher:
status:	Confirmed → In Progress

David TARDIVEL (david-tardivel) on 2016-09-15

Changed in watcher:
milestone:	ocata-rc-final → newton-rc-final

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-16: Fix merged to watcher (master)

Reviewed: https://review.openstack.org/370938
Committed: https://git.openstack.org/cgit/openstack/watcher/commit/?id=b6ab86c45a428243d41c0079e6ddc504a3aa96b5
Submitter: Jenkins
Branch: master

commit b6ab86c45a428243d41c0079e6ddc504a3aa96b5
Author: David TARDIVEL <email address hidden>
Date: Thu Sep 15 17:16:38 2016 +0200

Remove group_by statement in metric queries

ceilosca seems to not correctly handle group_by statement.
I remove it, because we don't need it for now.

Change-Id: Ie18551ba5a6bd155f9f4ae9c29c25b379affe019
Closes-Bug: #1623983

Changed in watcher:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-17: Fix included in openstack/watcher 0.32.0

This issue was fixed in the openstack/watcher 0.32.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.