Transient IOErrors with blob access and Zeo
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
ZODB |
New
|
Undecided
|
Unassigned |
Bug Description
Sometimes, a accessing a blob fails with the traceback below.
The bug cannot be reproduced reliably.
The blobs in question exist, this has been validated before. They have not been modified lately.
We noticed that the bug does not appear for several days after a restart of zope.
The /tmp/ directory has plenty of free space and plenty of free inodes.
Here is the traceback:
1342617765.
Traceback (innermost last):
Module ZPublisher.Publish, line 127, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 47, in call_object
Module project.
Module plone.app.
Module plone.app.
Module plone.app.
Module ZODB.blob, line 156, in open
Module ZODB.blob, line 307, in __init__
IOError: [Errno 2] No such file or directory: '/tmp/tmpke0dAV'
Some things I noticed:
The method openBlob in plone.app.
From my understanding of ZODB.blob and some experimentation on the system, the "/tmp/" directory is ONLY used when the blob object has _p_jar set to None. From my understand this only happens with a fresh Blob object that has never been persisted.
I am going to add traceback logging to plone.app.
If anybody has an idea what else to look for, I'd be very grateful.
The stacktrace has not helped alot but using sentry for logging helped and gave me an idea.
I can now reproduce the error on two of my four zeoclients. I can confirm that the objects have not been modified.
I noticed that the files ALWAYS the same tmpfile in the variable _p_blob_ uncommitted.
It does not change between requests, only between the instances.