The "slow load" issue is a cache locality problem. The following computation:
>>> [y for x, y in root.tags._tagid_to_obj.items() ... if y.community == 'nonesuch'] []
takes many seconds (likely the 30-40 originally reported) on a cold ZODB cache, but is nearly instantaneous on a hot cache.
Fixing this correctly likely involves generating sample data for profiling..
Here is some data, drawn from production:
>>> len(root.tags._tagid_to_obj) 37666 >>> len(root.tags._item_to_tagids) 7435 >>> len(root.tags._name_to_tagids) 4353 >>> len(root.tags._user_to_tagids) 500 >>> len(set([y.community for x, y in root.tags._tagid_to_obj.items()])) 384
The "slow load" issue is a cache locality problem. The following
computation:
>>> [y for x, y in root.tags. _tagid_ to_obj. items()
... if y.community == 'nonesuch']
[]
takes many seconds (likely the 30-40 originally reported) on a cold ZODB
cache, but is nearly instantaneous on a hot cache.
Fixing this correctly likely involves generating sample data for profiling..
Here is some data, drawn from production:
>>> len(root. tags._tagid_ to_obj) tags._item_ to_tagids) tags._name_ to_tagids) tags._user_ to_tagids) [y.community for x, y in root.tags. _tagid_ to_obj. items() ]))
37666
>>> len(root.
7435
>>> len(root.
4353
>>> len(root.
500
>>> len(set(
384