[DM] Garanteed Lifetime for Volatile Attributes
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| ZODB |
Won't Fix
|
Undecided
|
Unassigned | ||
Bug Description
2. Introduction
Volatile attributes have several interesting use cases
* cache the result of expensive computations
* store unpicklable resource which is object and connection specific
o The primary example is the database connection in 'Shared.
* store acquisition dependent information for use in an acquisition-less context
o The primary example has been the use of '_v_skindata' in the CMF 'Skinnable' implementation
* store temporary control information relevant for object processing further down in the
o hierarchy. The primary example is the use of '_v_is_cp' in 'Archetypes' to differenciate the use of 'manage_afterAdd' for either a copy or a move.
Apart from the first use case, all other use cases require a garanteed lifetime.
The ZODB does not garantee any lifetime for volatile attributes. Fortunately, for the use cases, the current ZODB implementation is quite conservative with objects. Usually, it deactivates objects only at transaction or savepoint boundaries and while the loss at a savepoint boundary occasionally breaks one of the use cases above, it happens rarely enough that volatile variables are still in use for these use cases. Sometimes, however, nasty and apparently non-deterministic errors occur due to unexpected loss of a volatile variable.
There are alternative solutions for all affected use cases; however much more complex than volatile attributes. For example the use of '_v_skindata' has been replaced in CMF by a module level registry indexed by id and thread identifier. As the data references persistent objects, it must not be used by a different connection. As in the next request the thread might be associated with a different connection, the thread specific data must be flushed at the end of the request which prevents the use of some caching optimizations. Furthermore, the current implementation breaks if several 'Skinnable' objects are used in the same request (we have a use case for this).
Therefore, I propose to implement a limited lifetime garantee for volatile attributes.
3. Feature
The proposed solution gives applications a means to indicate that an object should only be deactivated implicitely at transaction boundaries. We call such objects sticky as they stick more to the cache than normal objects. Stickyness does not affect explicit deactivation (by the application). As volatile attributes vanish implicitely only through deactivation, this effectively gives the volatile attributes of sticky objects a garanteed lifetime until the next transaction boundary. This is enough for all use cases listed above.
Cache garbage collection is slightly modified to distinguish between normal collection and strong collection. Normal collection does not deactivate sticky objects while strong collection can deactivate sticky objects as well.
As outlined above, strong collection is used at transaction boundaries while normal collection is used everywhere else.
Stickyness is declared by giving a new persistent attribute '_p_sticky' a true value. For efficiency reasons, '_p_sticky' is maintained at "C" level inside the header of persistent objects. Class level stickyness is taken care of during object construction. A later change of the class stickyness does not affect already created instances. 'del instance._p_sticky' will restore the class level stickyness (if any).
The cache garbage collection method gets a new parameter 'strong'. It will invalidate sticky objects only when 'strong' has a true value.
A ZODB Connection will call the cache garbage collection with 'strong=True' only at transaction boundaries.
4. Example Use Cases
We distinguish two use cases
* class level stickyness
o The class knows that is uses volatile variables in a way requiring transaction wide lifetime and therefore declares stickyness for all its instances. This use case supports e.g. the use in CMF 'Skinnable' and in Zopes database adapters.
* instance level stickyness
o The application needs to temporarily protect a volatile variable. This use case supports e.g. the use of '_v_is_cp' in Archetypes.
5. Risks
* Handling class level stickyness correctly might be a bit tricky.
* The behavior of instance stickyness when the class stickyness changes (the stickyness
* of existing instances does **NOT** change) might be surprising.
* When an application sets an instance level stickyness and later does not reset it,
* the lifetime of the stickyness declaration extends to at least the transaction boundary as the instance will not be deactivated (automatically) before that time. However, it may live arbitrarily longer. It survives deactivation and dies only when the instance is flushed from the connection cache completely. This might be surprising but almost surely will remain unnoticed.
| Andreas Jung (ajung) wrote | #1 |
| Jim Fulton (jim-zope) wrote | #2 |
| Changed in zodb: | |
| status: | New → Won't Fix |
| Simon Pamies (s-pamies) wrote | #3 |
| Jim Fulton (jim-zope) wrote Re: [Bug 374751] Re: [DM] Garanteed Lifetime for Volatile Attributes | #4 |
This is a feature request, not a bug.
See: https:/