Comment 2 for bug 113923

Revision history for this message
Jim Fulton (jim-zope) wrote : Re: [Bug 113923] cache GC interacts badly with multiple threads

On May 10, 2007, at 10:38 PM, Tim McLennan wrote:

> Public bug reported:
>
> Okay, I somehow managed to lose my first version of what I was
> going to
> say here.
>
> Basically, when you run with a cache-size of 0, (although it is
> theoretically possible to occur with other sizes as well), you can run
> into problems if the garbage collection for the cache is run while you
> are using objects in that thread.

We generally try to avoid this ...

> Unfortunately the db.open call will
> runs the GC on the caches for all connections when you create a new
> connection.

Yipes. It shouldn't do that. Hm.

>
> The sorts of errors that can occur can be quite confusing. For
> example take the code for the persistent mapping __getstate__
> routine (as called eventually by Transaction.commit())
> def __getstate__(self):
> state = {}
> state.update(self.__dict__)
> state['_container'] = state['data']
> del state['data']
> return state
>
> if the object is ghostified before the self.__dict__ reference then
> you'll get a KeyError('data') exception in the 3rd line.

Interesting. I'll need to review the commit code. I would think
that the object would still be marked as changed at this point so it
wouldn't get ghostified, but the dance during commit may have already
marked it as saved -- although that would seem a bit odd.

> Another case is when you store a reference to a mutable (non-
> persistent)
> subobject and then access it.
>
> l = self.list
> l.append([xyz])
> self._p_changed = 1
>
> then if self is ghostified before settting _p_changed then
> self.list !=
> l and so you won't actually change the object like you mean to.
>
> In fact if you're really unlucky you can get the case where:
> self.member == self.member will be false.

I seem to remember a discussion a while back about the proper order
to do this sort of thing, although I think it shouldn't be this
delicate.

This is a very good argument to not do GC on a connection while
application code is using it.

Thanks for a very interesting and informative bug report. I'll dig
into this very soon -- hopefully today or this weekend.

Jim

--
Jim Fulton mailto:<email address hidden> Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org