sort against documents should be done on server

Bug #1204609 reported by Tong Li
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Medium
Tong Li

Bug Description

in impl_mongodb.py, sort was done after documents returned from the db calls. sort should be done on the server side. Here is where the problem was found.

    def get_users(self, source=None):
        """Return an iterable of user id strings.

        :param source: Optional source filter.
        """
        q = {}
        if source is not None:
            q['source'] = source
        return sorted(self.db.user.find(q).distinct('_id'))

Notice the sorted method call at the end.

same problem in get_projects method.

Julien Danjou (jdanjou)
Changed in ceilometer:
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Julien Danjou (jdanjou)
Tong Li (litong01)
Changed in ceilometer:
assignee: Julien Danjou (jdanjou) → Tong Li (litong01)
Changed in ceilometer:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/38537
Committed: http://github.com/openstack/ceilometer/commit/4514eb87675ed9cf46d9fd4397727059b7881a37
Submitter: Jenkins
Branch: master

commit 4514eb87675ed9cf46d9fd4397727059b7881a37
Author: Tong Li <email address hidden>
Date: Thu Jul 25 10:33:55 2013 -0400

    calling distinct on _id field against a collection is slow

    To optimize the code, distinct method should be avoided especially
    against _id field. Sort should also be done on the server side to
    take advantage of generator to avoid using up memory for large
    data set.

    Change-Id: Ib417f73e2238a5d9865a2a1a9990e891b0f7a24c
    fixes: bug #1204607
    fixes: bug #1204609

Changed in ceilometer:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: none → havana-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: havana-3 → 2013.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.