Convert blog listing to get_catalog_batch

Bug #1518973 reported by Paul Everitt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL4
Won't Fix
Medium
Carlos de la Guardia

Bug Description

Make sure the tests get refactored correctly.

Revision history for this message
Carlos de la Guardia (cguardia) wrote :

Added a proposed fix on branch get_catalog_batch.

Changed in karl4:
status: New → Fix Committed
status: Fix Committed → In Progress
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Shane has a neat way to look at object loads. Add a ?__debugload__ to the end of the URL and you'll get, in your karl.log file, a dump of how many of each kind of object was loaded during the request.

For a small blog, the container version loaded 165 objects:

165 ZODB objects loaded after traversal
      38 <class 'karl.content.models.blog.BlogEntry'>
      25 <type 'BTrees.OOBTree.OOBTree'>
...etc.

and the catalog version loaded 7,965 objects:

7976 ZODB objects loaded after traversal
    4634 <type 'BTrees.IFBTree.IFSet'>
    2867 <type 'BTrees.IFBTree.IFTreeSet'>
     228 <type 'BTrees.IOBTree.IOBucket'>
...etc.

For a big blog, the container version loaded:

1679 ZODB objects loaded after traversal
    1509 <class 'karl.content.models.blog.BlogEntry'>
      75 <type 'BTrees.OOBTree.OOBucket'>
      25 <type 'BTrees.OOBTree.OOBTree'>

...and the catalog version loaded:

8 ZODB objects loaded during traversal
       3 <type 'BTrees.OOBTree.OOBTree'>
       1 <class 'persistent.mapping.PersistentMapping'>
       1 <type 'BTrees.OOBTree.OOBucket'>
       1 <class 'karl.models.site.Site'>
       1 <class 'karl.models.community.CommunitiesFolder'>
       1 <class 'karl.models.community.Community'>
14161 ZODB objects loaded after traversal
    6230 <type 'BTrees.IFBTree.IFSet'>
    2867 <type 'BTrees.IFBTree.IFTreeSet'>
    2244 <type 'BTrees.IOBTree.IOBucket'>
    1509 <class 'karl.content.models.blog.BlogEntry'>
     968 <type 'BTrees.IIBTree.IIBucket'>
     113 <type 'BTrees.OIBTree.OIBucket'>

As the email from Shane notes, it is possible that I needed to mutate the catalog to get a more accurate reading. Carlos, perhaps we need to let you do the driving on karlstaging on some things, as I'm a bit of a bottleneck (and, you might want to experiment.)

One other thing to note...the blog view has a portlet which gets counts of earlier months. That probably defeats any attempt we have at only looking at 50 things.

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

The only next step I can imagine is for Carlos to read and process Shane's email about the ?__debugload__ flag, plus the caveat about mutating the catalog, then trying some experiments on karlstaging. It should take an hour or less to prove/disprove that the catalog helps/hurts us on small/large blogs/folders.

Moving to January, less urgent.

Changed in karl4:
milestone: 013 → 015
Revision history for this message
Carlos de la Guardia (cguardia) wrote :

I made several tries with different parameters. The catalog batch loads way more objects on first load, and just a bit more objects than the container batch once the catalog is cached. It also is slower. I guess we'll need to find some other to optimize these views.

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Let's close this one as infeasible. Thanks for the investigation.

Changed in karl4:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.