API server: improve list performence

[Édouard Thuleau]

    2. In resource-listing, if not admin-request a call to read from DB for obj_uuids is made with id-perms specified.
/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py(924)http_resource_list()
-> filters, req_fields)
  /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py(2265)_list_collection()
-> obj_type, obj_ids_list, obj_fields)
  /usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py(1795)dbe_read_multi()
-> obj_fields)
> /usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py(333)object_read()

       However vnc_cassandra.py object_read reads in all properties + refs if field_names doesnt have any children or backref.
       Add one more if check to see if only props have been specified in field_names, read from cassandra with columns=[prop:+x for x in field_names]

so need to change...
321 if (field_names is None or
322 (set(field_names) & (obj_class.backref_fields | obj_class.children_fields))):
323 # atleast one backref/children field is needed
324 obj_rows = obj_uuid_cf.multiget(obj_uuids,
325 column_count=self._MAX_COL,
326 include_timestamp=True)
327 else: # ignore reading backref + children columns
(Pdb)
328 obj_rows = obj_uuid_cf.multiget(obj_uuids,
329 column_start='d',
330 column_count=self._MAX_COL,
331 include_timestamp=True)
...
to (need to handle propl: case)
        if (field_names is None or
            (set(field_names) & (obj_class.backref_fields | obj_class.children_fields))):
            # atleast one backref/children field is needed
            obj_rows = obj_uuid_cf.multiget(obj_uuids,
                                       column_count=self._MAX_COL,
                                       include_timestamp=True)
        elif not set(field_names) & (obj_class.ref_fields):
            # specific props have been asked fetch exactly those
            columns = []
            for fname in field_names:
                if fname in obj_class.prop_fields:
                    columns.append('prop:'+fname)
                else:
                    columns.append(fname)
            obj_rows = obj_uuid_cf.multiget(obj_uuids,
                columns=set(columns)|set(['fq_name', 'parent_type']),
                column_count=self._MAX_COL,
                include_timestamp=True)
        else: # ignore reading backref + children columns
            obj_rows = obj_uuid_cf.multiget(obj_uuids,
                                       column_start='d',
                                       column_count=self._MAX_COL,
                                       include_timestamp=True)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20525
Submitter: ?douard Thuleau (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20525
Committed: http://github.org/Juniper/contrail-controller/commit/6835c37406d66773bc567aba4ecb550b6eea6fe0
Submitter: Zuul
Branch: master

commit 6835c37406d66773bc567aba4ecb550b6eea6fe0
Author: Édouard Thuleau <email address hidden>
Date: Fri May 20 12:15:28 2016 +0200

[VNC API server] Improving list resources

When a resource type is listed, a multi-read on cassandra is done
on each resource. That patch optimizes that multi-read to fetch only
specified resources properties when it's feasible.

Change-Id: If3bf44eeb0330a3d8d48f34531645db6537a0d8b
Closes-Bug: #1584000
Partial-Bug: #1466777