Hbase metadata query return wrong result

[Feilong Wang]

Based on current implement of Hbase support in Ceilometer, it's trying to compare the metadata value with the data stored in table "resource" when user query a specific meter name. See: https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_hbase.py#L441 and https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_mongodb.py#L535

It can be reproduced by running unit test case "test_list_meters_scenarios" by update the fake data and a new test case.

                counter.Counter(
                    'meter.test',
                    'cumulative',
                    '',
                    1,
                    'user-id',
                    'project-id',
                    'resource-id',
                    timestamp=datetime.datetime(2012, 7, 2, 10, 40),
                    resource_metadata={'display_name': 'test-server',
                                       'tag': 'self.counter'}),
                counter.Counter(
                    'meter.test',
                    'cumulative',
                    '',
                    3,
                    'user-id',
                    'project-id',
                    'resource-id',
                    timestamp=datetime.datetime(2012, 7, 2, 11, 40),
                    resource_metadata={'display_name': 'test-server',
                                       'tag': 'self.counter999'}),

    def test_list_meters_query_metadata(self):
        data = self.get_json('/meters/meter.test',
                             q=[{'field': 'metadata.tag',
                             'op': 'eq',
                             'value': 'self.counter999'}]
                             )
        self.assertEquals(1, len(data))

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/39025

[Feilong Wang (flwang)]

Besides, what I was working on defect: https://bugs.launchpad.net/ceilometer/+bug/1203699. The test case for Hbase will run into below issue. So I think there is a bug for the dict metadata query of Hbase. I will open a new bug to track it.

FAIL: tests.api.v2.test_list_meters_scenarios.TestListMeters.test_list_meters_with_dict_metadata(hbase)
tags: worker-0
----------------------------------------------------------------------
pythonlogging:'': {{{
Server-side error: "u'f:r_properties.extra_prop'". Detail:
Traceback (most recent call last):

  File "/home/flwang/MyWorkSpace/workspace_community/ceilometer/.tox/py27/local/lib/python2.7/site-packages/wsmeext/pecan.py", line 70, in callfunction
    result = f(self, *args, **kwargs)

  File "/home/flwang/MyWorkSpace/workspace_community/ceilometer/ceilometer/api/controllers/v2.py", line 476, in get_all
    for e in pecan.request.storage_conn.get_samples(f, limit=limit)

  File "/home/flwang/MyWorkSpace/workspace_community/ceilometer/ceilometer/storage/impl_hbase.py", line 444, in get_samples
    if resource['f:r_' + k.split('.', 1)[1]] != v:

KeyError: u'f:r_properties.extra_prop'
}}}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/39025
Committed: http://github.com/openstack/ceilometer/commit/ea76c9be8c6a84cd78c324bc67a3665bf6d464b2
Submitter: Jenkins
Branch: master

commit ea76c9be8c6a84cd78c324bc67a3665bf6d464b2
Author: Fei Long Wang <email address hidden>
Date: Sun Jul 28 16:45:29 2013 +0800

    Fixes Hbase metadata query return wrong result

    Based on current implement of Hbase metadata query in Ceilometer,
    it's trying to compare the metadata value with the metadata stored
    in table "resource" instead of table "meter", when user query a
    specific meter name. As a result, the query result will be wrong.
    See:
    https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_hbase.py#L441
    https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_mongodb.py#L535

    Fixes bug 1205759

    Change-Id: I3f8168fa5371985ac135b0a9d41e76d7bc71c4a7