Querying measurments with both merge and group_by flags return bad result.

[Ryan Bak]

If you make a monasca measurments query with with '--merge' and '--group_by dim', only a few data points are returned for each dimension set. A statistics query with the same parameters will correctly return all data.

An example failing query:
monasca measurement-list rabbitmq.node.mem_used --merge 2017-01-22T19:00:00.000Z --endtime 2017-01-23T19:00:00.000Z --group_by region

[Ryan (ryan-brandt)]

I've not had a chance to bring up an environment to check this yet, can you include the result? Also, I presume this was java and vertica, can you confirm?

[Ryan Bak (ryanmbak)]

Good point. Yes this is java and vertica.

I'm including both statistics and measurements below. I've narrowed the time range to make it more manageable. The statistics query (second) has period=30 seconds so I would expect that to be roughly the same as a measurements list.

monasca measurement-list rabbitmq.node.mem_used --merge 2017-01-23T19:00:00.000Z --endtime 2017-01-23T20:00:00.000Z --group_by node

{"links":[{"rel":"self","href":"https://bfd-api.os.cloud.twc.net:8070/v2.0/metrics/measurements?group_by=node&start_time=2017-01-23T19%3A00%3A00.000Z&merge_metrics=True&name=rabbitmq.node.mem_used&end_time=2017-01-23T20%3A00%3A00.000Z"}],"elements":[{"id":"rabbit@bfd02-control-001","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-control-001"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.319Z",5.5726476E8,{}]]},{"id":"rabbit@bfd02-control-002","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-control-002"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.319Z",1.277690352E9,{}]]},{"id":"rabbit@bfd02-control-003","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-control-003"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.319Z",1.19411384E9,{}]]},{"id":"rabbit@bfd02-trove-001","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-trove-001"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.398Z",9.061176E7,{}]]},{"id":"rabbit@bfd02-trove-002","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-trove-002"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.398Z",8.7207944E7,{}]]},{"id":"rabbit@bfd02-trove-003","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-trove-003"},"columns":["timestamp","value","value_meta"],"measurements":[["2017-01-23T19:59:36.398Z",9.7235056E7,{}]]}]}

monasca metric-statistics rabbitmq.node.mem_used --merge AVG 2017-01-23T19:00:00.000Z --endtime 2017-01-23T19:10:00.000Z --group_by node --period 30

{"links":[{"rel":"self","href":"https://bfd-api.os.cloud.twc.net:8070/v2.0/metrics/statistics?merge_metrics=True&name=rabbitmq.node.mem_used&start_time=2017-01-23T19%3A00%3A00.000Z&period=30&group_by=node&end_time=2017-01-23T19%3A10%3A00.000Z&statistics=AVG"}],"elements":[{"id":"rabbit@bfd02-control-001","name":"rabbitmq.node.mem_used","dimensions":{"node":"rabbit@bfd02-control-001"},"columns":["timestamp","value","value_meta"],"statistics":[["2017-01-23T19:00:00.000Z",5.54403944E8],["2017-01-23T19:00:30.000Z",5.478292E8],["2017-01-23T19:01:00.000Z",5.53139536E8],["2017-01-23T19:01:30.000Z",5.50919512E8],["2017-01-23T19:02:00.000Z",5.524032E8],["2017-01-23T19:02:30.000Z",5.49284856E8],["2017-01-23T19:03:00.000Z",5.55144408E8],["2017-01-23T19:03:30.000Z",5.50048784E8],["2017-01-23T19:04:00.000Z",5.5593384E8],["2017-01-23T19:04:30.000Z",5.50711552E8],["2017-01-23T19:05:00.000Z",5.5611228E8],["2017-01-23T19:05:30.000Z",5.5303204E8],["2017-01-23T19:06:00.000Z",5.5768724E8],["2017-01-23T19:06:30.000Z",5.54984576E8],["2017-01-23T19:07:00.000Z",5.55675576E8...

[Ryan (ryan-brandt)]

Thanks for the additional data, I will try to test this today. I'm classifying this as low for now, since --merge and --group_by aren't meant to be used together.

[Ryan Bak (ryanmbak)]

Sounds good. I'm fine with low importance, since it obviously took us so long to notice, but be aware that the grafana plugin hits this bug if you aren't using statistics.