Invalid literal for int() when processing a large number of objects

[Rahul Kumar]

Hi All,

We have a plone site with ZODB 3.8.3 with large number of objects. Whenever we run a report which processes a large number of objects, we get the following error:

2018-05-04T01:57:12 ERROR ZODB.Connection Couldn’t load state for 0x030c8158
Traceback (most recent call last):
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/Connection.py”, line 811, in setstate_
_ self.setstate(obj)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/Connection.py”, line 870, in setstate
_ self.reader.setGhostState(obj, p)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/serialize.py”, line 604, in setGhostState_
_ state = self.getState(pickle)_
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/serialize.py”, line 597, in getState_
_ return unpickler.load()_
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/serialize.py”, line 471, in persistent_load
_ return self.load_oid(reference)_
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/serialize.py”, line 537, in load_oid_
_ return self.conn.get(oid)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZODB/Connection.py”, line 244, in get_
_ p, serial = self.storage.load(oid, self.version)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py”, line 712, in load
_ return self.loadEx(oid, version)[:2]_
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py”, line 740, in loadEx_
_ self.cache.store(oid, ver, tid, None, data)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZEO/cache.py”, line 136, in call_
_ return self.func(inst, *args, **kw)_
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZEO/cache.py”, line 557, in store_
_ nfreebytes = self.makeroom(size)
_ File “/usr/local/Plone/buildout-cache/eggs/ZODB3-3.8.3-py2.4-linux-x86_64.egg/ZEO/cache.py”, line 370, in makeroom
_ size = int(status)_
ValueError: invalid literal for int():
------
2018-05-04T01:57:12 ERROR Zope.SiteErrorLog 1525413432.290.433195371895 https://ocwcms.mit.edu/@@report_generator
Traceback (innermost last):
_ Module ZPublisher.Publish, line 119, in publish_
_ Module ZPublisher.mapply, line 88, in mapply_
_ Module ZPublisher.Publish, line 42, in call_object_
_ Module Shared.DC.Scripts.Bindings, line 313, in call_
_ Module Shared.DC.Scripts.Bindings, line 350, in bindAndExec
_ Module Products.PageTemplates.PageTemplateFile, line 129, in exec
_ Module Products.PageTemplates.PageTemplate, line 98, in pt_render_
_ Module zope.pagetemplate.pagetemplate, line 117, in pt_render_
_ Module zope.tal.talinterpreter, line 271, in call_
_ Module zope.tal.talinterpreter, line 346, in interpret_
_ Module zope.tal.talinterpreter, line 891, in do_useMacro_
_ Module zope.tal.talinterpreter, line 346, in interpret_
_ Module zope.tal.talinterpreter, line 536, in do_optTag_tal_
_ Module zope.tal.talinterpreter, line 521, in do_optTag_
_ Module zope.tal.talinterpreter, line 516, in no_tag_
_ Module zope.tal.talinterpreter, line 346, in interpret_
_ Module zope.tal.talinterpreter, line 949, in do_defineSlot_
_ Module zope.tal.talinterpreter, line 346, in interpret_
_ Module zope.tal.talinterpreter, line 536, in do_optTag_tal_
_ Module zope.tal.talinterpreter, line 521, in do_optTag_
_ Module zope.tal.talinterpreter, line 516, in no_tag_
_ Module zope.tal.talinterpreter, line 346, in interpret_
_ Module zope.tal.talinterpreter, line 586, in do_setLocal_tal_
_ Module zope.tales.tales, line 696, in evaluate_
_ - URL: report_generator_
_ - Line 27, Column 4_
_ - Expression: <PathExpr standard:‘view/getReportData’>_
_ - Names:_
_ {‘container’: <PloneSite at /Plone>,_
_ ‘context’: <PloneSite at /Plone>,_
_ ‘default’: <object object at 0x2ad0f9099200>,_
_ ‘here’: <PloneSite at /Plone>,_
_ ‘loop’: {},_
_ ‘nothing’: None,_
_ ‘options’: {‘args’: ()},_
_ ‘repeat’: <Products.PageTemplates.Expressions.SafeMapping object at 0x2aaacafb0950>,_
_ ‘request’: <HTTPRequest, URL=https://ocwcms.mit.edu/@@report_generator>,_
_ ‘root’: ,_
_ ‘template’: <ImplicitAcquirerWrapper object at 0x2aaac9eba810>,_
_ ‘traverse_subpath’: [],_
_ ‘user’: <PloneUser ‘rkumar174’>,_
_ ‘view’: <Products.Five.metaclass.HtmlReportGeneratorView object at 0x2aaac9ea26d0>,_
_ ‘views’: <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0x2aaacafb1bd0>}_
_ Module zope.tales.expressions, line 217, in call_
_ Module Products.PageTemplates.Expressions, line 163, in eval
_ Module Products.PageTemplates.Expressions, line 125, in render_
_ Module ocw.publishing.browser.reports.html_report_generator, line 39, in getReportData_
_ Module ocw.publishing.browser.reports.html_report_generator, line 82, in getCourseSectionResourceDetailsReport_
_ Module ocw.publishing.browser.reports.CourseSectionResourceDetails, line 72, in processReport_
_ Module ocw.publishing.browser.reports.CourseSectionResourceDetails, line 93, in getDataFromDepartment
_ Module ZODB.Connection, line 811, in setstate_
_ Module ZODB.Connection, line 870, in setstate
_ Module ZODB.serialize, line 604, in setGhostState_
_ Module ZODB.serialize, line 597, in getState_
_ Module ZODB.serialize, line 471, in persistent_load
_ Module ZODB.serialize, line 537, in load_oid_
_ Module ZODB.Connection, line 244, in get_
_ Module ZEO.ClientStorage, line 712, in load_
_ Module ZEO.ClientStorage, line 740, in loadEx_
_ Module ZEO.cache, line 136, in call_
_ Module ZEO.cache, line 557, in store_
_ Module ZEO.cache, line 370, in makeroom
ValueError: invalid literal for int():

If we segregate the report in 5-6 reports we don’t get this error. We have also noticed that the /temp folder size increases very much. Can someone please suggest a solution for this?

Thanks,
Rahul