cardinality of storage.models.Resource.meter attribute is one-per-sample

[Eoghan Glynn]

At least in the mongodb and sqlalchemy cases (and most likely for all storage drivers), the meter attribute of the Resource model has an entry for every single sample associated with the given resource.

This results in an extremely long and repetitive list of meter (name,type,unit) tuples, which is then promptly discarded as the meter link hrefs in the API Resource representation are recreated from separate get_meters calls on the storage driver (one per unique resource ID in the result set for the GET /v2/resources query).

So we've created a huge scaling issue, for no good reason.

We should either:

1. just stop populating the Resource.meter in the storage model, as it's currently unused, contains much redundant information, and is expensive to construct in memory

or:

2. ensure the Resource.meter list is dupe-free and suppress the multiple calls into get_meters for each resource ID to create the resource links

[Julien Danjou (jdanjou)]

Sounds like we should go for 1.

[gordon chung (chungg)]

i'm currently working on filtering out name, type, unit attributes into a separate table (relating to bug 1211985). it might help implement 2nd option if required. that said, i'd prefer option 1 as well. i don't really have a use case to keep Resource.meter right now.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/65671

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/65671
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=7c4c0e8fc098fbadec2e30ea2bdf1068b5b4288a
Submitter: Jenkins
Branch: master

commit 7c4c0e8fc098fbadec2e30ea2bdf1068b5b4288a
Author: Eoghan Glynn <email address hidden>
Date: Thu Jan 9 11:11:43 2014 +0000

    Remove redundant meter (name,type,unit) tuples from Resource model

    Fixes bug 1267162

    Previously, a list of meter (name,type,unit) tuples with per-sample
    cardinality was added to the Resource model by the storage driver
    get_resource method.

    However this potential huge and highly repetitive list was then
    promptly discarded by the API layer, as the meter links in the API
    Resource representation are populated via subsequent calls into
    the storage driver get_meters method.

    Now, we avoid the scaling issue by suppressing the creation of
    this unused meter list.

    Change-Id: I18dee7552836c724798ea1340e2e933c09d8a294

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/65947

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/65947
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=ef71dc6a11ab624e756bfb61ec974e9f6096bc30
Submitter: Jenkins
Branch: stable/havana

commit ef71dc6a11ab624e756bfb61ec974e9f6096bc30
Author: Eoghan Glynn <email address hidden>
Date: Thu Jan 9 11:11:43 2014 +0000

    Remove redundant meter (name,type,unit) tuples from Resource model

    Fixes bug 1267162

    Previously, a list of meter (name,type,unit) tuples with per-sample
    cardinality was added to the Resource model by the storage driver
    get_resource method.

    However this potentially huge and highly repetitive list was then
    promptly discarded by the API layer, as the meter links in the API
    Resource representation are populated via subsequent calls into
    the storage driver get_meters method.

    Now, we avoid the scaling issue by suppressing the creation of
    this unused meter list.

    Change-Id: I18dee7552836c724798ea1340e2e933c09d8a294
    (cherry picked from commit 7c4c0e8fc098fbadec2e30ea2bdf1068b5b4288a)