resource-list doesn't need to join sample table if no timestamp is specified (mysql backend)

Bug #1509677 reported by ZhiQiang Fan on 2015-10-24
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Undecided
ZhiQiang Fan
Kilo
Fix Released
Undecided
Unassigned

Bug Description

for sqlalchemy implementation, currently, even there is no timestamp is specified, we still inner join sample table, this is actually not needed

https://github.com/openstack/ceilometer/blob/b34865f80818165187552e7feca4ead2e61a30d3/ceilometer/storage/impl_sqlalchemy.py#L434

ZhiQiang Fan (aji-zqfan) on 2015-10-24
Changed in ceilometer:
assignee: nobody → ZhiQiang Fan (aji-zqfan)

Fix proposed to branch: master
Review: https://review.openstack.org/239183

Changed in ceilometer:
status: New → In Progress
Rohit Jaiswal (rohit-jaiswal-3) wrote :

IIUC the inner join is still needed to filter out resources that do not have any samples, otherwise [1] will loop over resources with no samples and generate 2 queries per resource that dont return anything..

[1] https://github.com/openstack/ceilometer/blob/b34865f80818165187552e7feca4ead2e61a30d3/ceilometer/storage/impl_sqlalchemy.py#L440

ZhiQiang Fan (aji-zqfan) wrote :

@Rohit

Yes, that would happend when cfg.CONF.sql_expire_samples_only is true, then there will be resources without any samples, I will update the patch to reflect such scenario

Thanks!

Reviewed: https://review.openstack.org/239183
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=a4f442940551a8a38e26e132c7efa766226daa15
Submitter: Jenkins
Branch: master

commit a4f442940551a8a38e26e132c7efa766226daa15
Author: ZhiQiang Fan <email address hidden>
Date: Sat Oct 24 09:30:53 2015 -0600

    avoid unnecessary inner join in get_resources() for SQL backend

    To get distinct resource ids, we do a query on resource table which
    inner join sample table, and apply filters on it.

    Note that when sql_expire_samples_only is enabled, there will be
    some resources without any sample, in such case we must use inner
    join to avoid wrong result, no matter if there is a timestamp filter
    or not.

    But that option is disabled by default, so when there is no timestamp
    filters, the inner join is unnecessary, we should avoid it to save
    some RAM/CPU cost.

    Change-Id: If85dbea15d42d42c6b0be7402c06f258e278b2eb
    Closes-Bug: #1509677

Changed in ceilometer:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/242962
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=14586b2eac909dd76431e4eb9ff00b8f5a4c2d0c
Submitter: Jenkins
Branch: stable/kilo

commit 14586b2eac909dd76431e4eb9ff00b8f5a4c2d0c
Author: ZhiQiang Fan <email address hidden>
Date: Sat Oct 24 09:30:53 2015 -0600

    avoid unnecessary inner join in get_resources() for SQL backend

    To get distinct resource ids, we do a query on resource table which
    inner join sample table, and apply filters on it.

    Note that when sql_expire_samples_only is enabled, there will be
    some resources without any sample, in such case we must use inner
    join to avoid wrong result, no matter if there is a timestamp filter
    or not.

    But that option is disabled by default, so when there is no timestamp
    filters, the inner join is unnecessary, we should avoid it to save
    some RAM/CPU cost.

    Change-Id: If85dbea15d42d42c6b0be7402c06f258e278b2eb
    Closes-Bug: #1509677
    (cherry picked from commit a4f442940551a8a38e26e132c7efa766226daa15)

tags: added: in-stable-kilo

Reviewed: https://review.openstack.org/242959
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=81dded73ce81b9c62e196003f72451338db65649
Submitter: Jenkins
Branch: stable/liberty

commit 81dded73ce81b9c62e196003f72451338db65649
Author: ZhiQiang Fan <email address hidden>
Date: Sat Oct 24 09:30:53 2015 -0600

    avoid unnecessary inner join in get_resources() for SQL backend

    To get distinct resource ids, we do a query on resource table which
    inner join sample table, and apply filters on it.

    Note that when sql_expire_samples_only is enabled, there will be
    some resources without any sample, in such case we must use inner
    join to avoid wrong result, no matter if there is a timestamp filter
    or not.

    But that option is disabled by default, so when there is no timestamp
    filters, the inner join is unnecessary, we should avoid it to save
    some RAM/CPU cost.

    Change-Id: If85dbea15d42d42c6b0be7402c06f258e278b2eb
    Closes-Bug: #1509677
    (cherry picked from commit a4f442940551a8a38e26e132c7efa766226daa15)

tags: added: in-stable-liberty

This issue was fixed in the openstack/ceilometer 6.0.0.0b1 development milestone.

Thierry Carrez (ttx) on 2015-12-03
Changed in ceilometer:
status: Fix Committed → Fix Released

This issue was fixed in the openstack/ceilometer 5.0.1 release.

Liusheng (liusheng) on 2015-12-14
Changed in ceilometer:
milestone: none → mitaka-1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers