Product reload broken

[Stuart Bishop]

After a product reload, there is a fair chance that I'll get the following exception when accessing an object defined by that product:

Traceback (innermost last):
  Module ZPublisher.Publish, line 102, in publish
  Module Zope.App.startup, line 220, in commit
  Module ZODB.Transaction, line 233, in commit
  Module ZODB.Transaction, line 348, in _commit_objects
  Module ZODB.Connection, line 427, in commit
   - __traceback_info__: (('BTrees.IOBTree', 'IOBucket'), '\x00\x00\x00\x00\x00\x00\x00G', '')
ValueError: Cache values may only be in one cache.

I've received other errors, but the above is the most common and
I've received confirmation that at least one other person is receiving them.

Restarting Zope fixes everything.

This issue will need to be resolved before 2.7 final, or reload functionality removed.

[Stuart Bishop (stub)]

To clarify this, *manual* product refresh using the Product's Refresh tab works as expected. *automatic* product refresh fails.

This is still present in 2.7b3.

[Tim Peters (tim-one)]

Dieter Maurer may have found a good clue here:

http://mail.zope.org/pipermail/zope-dev/2003-December/021115.html

[Santi Camps (scamps)]

Still present in Zope 2.7.0 final

Refreshing the Product using ZMI also fixes everything

[Ian J Cottee (icottee)]

Any idea when/if this will be fixed? It happens everytime with some very very simple Archetypes I am working with and it's a pain (using zope-2.7.1-b2).

[Vahur Rebas (berlingo666)]

Is this issue getting anywhere? status still pending... or is it already fixed in some version?
Keep getting this error.
Product is python-class based with lots of ZPT-s
Using Zope 2.7.0 release.

1:
Traceback (innermost last):

    * Module ZPublisher.Publish, line 104, in publish
    * Module Zope.App.startup, line 221, in commit
    * Module ZODB.Transaction, line 233, in commit
    * Module ZODB.Transaction, line 348, in _commit_objects
    * Module ZODB.Connection, line 425, in commit
      __traceback_info__: (('BTrees.OOBTree', 'OOBTree'), '\x00\x00\x00\x00\x00\x00\x00\x8f', '')

ValueError: Cache values may only be in one cache.

2:
Traceback (innermost last):

    * Module ZPublisher.Publish, line 104, in publish
    * Module Zope.App.startup, line 221, in commit
    * Module ZODB.Transaction, line 233, in commit
    * Module ZODB.Transaction, line 348, in _commit_objects
    * Module ZODB.Connection, line 425, in commit
      __traceback_info__: (('Products.Transience.Transience', 'Increaser'), '\x00\x00\x00\x00\x00\x00\x00\x0b', '')

ValueError: Cache values may only be in one cache.

these 2 differ the most from others.. and I think I have got this error using automatic and manual refreshing

[Artur Zaprzala (zybi)]

I did some research and found the way to replicate this bug.
1. Refresh some product
2. Call REQUEST/SESSION (e.g. from PageTemplate)
(Tested with Zope 2.9 and 2.10)

And then I traced the culprit. When refresh is done, old PickleCache for connection is cleared and replaced with a new one. But temp_folder/session_data seems to be broken and some objects stay in the old cache. And when you referece such object again, ne w cache is trying to cache them, and you get:
ValueError: Cache values may only be in one cache.
Tracebacks from previous post are related to session_data attributes.

Anybody knows how to fix this?

[Artur Zaprzała (arturz)]

This bug was annoying me for many years. I'm happy to share the solution.

The problem was that when the PickleCache of Connection was replaced with a new one, the old one was still kept by the ObjectReader instance.

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Artur Zaprzała wrote:
> This bug was annoying me for many years. I'm happy to share the
> solution.
>
> The problem was that when the PickleCache of Connection was replaced
> with a new one, the old one was still kept by the ObjectReader instance.
>
> ** Attachment added: "A missing line"
> http://launchpadlibrarian.net/18373704/Connection.py.patch

I concur: this patch looks obviously right by inspection. The bug
itself should be against ZODB and not Zope2, however.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI7mcC+gerLs4ltQ4RAq0kAKCjAk3OZxaFvFNK374qbgI5ETrcVQCgig6s
yiS96aYt94cQQEuRhP9sHwY=
=g8CL
-----END PGP SIGNATURE-----

[Tres Seaver (tseaver)]

Committed to SVN:

- ZODB 3.4 branch: http://svn.zope.org/ZODB/branches/3.4/?rev=91949&view=rev
  (tagged for internal release 3.4.4, for use in putative Zope 2.8.10 release).

- ZODB 3.5 branch: http://svn.zope.org/ZODB/branches/3.5/?rev=91950&view=rev
  (note broken tests on branch prior to applying the patch / new test; this branch is
  not used by any release of Zope2 or Zope3).

- ZODB 3.6 branch: http://svn.zope.org/ZODB/branches/3.6/?rev=91952&view=rev
  (tagged for internal release 3.6.4, for use in putative Zope 2.9.10 release)

- ZODB 3.7 branch: http://svn.zope.org/ZODB/branches/3.7/?rev=91956&view=rev
  (tagged for internal release 3.7.3, for use in Zope 2.10.7 release).

- ZODB 3.8 branch: (commit pending either 3.8.1 release or permission to
  merge prior to that release). Ideally, this change would be included in 3.8.1.

- ZODB trunk: http://svn.zope.org/ZODB/trunk/?rev=91964&view=rev

[Tres Seaver (tseaver)]

This is the patch / test that I have been porting / applying.

[Artur Zaprzała (arturz)]

As for your patch, looking at ZODB 3.8 I didn't find a possibility for self._reader to not exist or be None. Did I miss something?

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Artur Zaprzała wrote:
> As for your patch, looking at ZODB 3.8 I didn't find a possibility for
> self._reader to not exist or be None. Did I miss something?

I think you are correct for the 3.8 branch: it alloacates the '_reader'
in '__init__'. I was applying the fix to older versions, as well,
where the '_reader' object is not created until the end of '_setDB',
*after* the call to ''_resetCache'.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI70ni+gerLs4ltQ4RAgmFAJ4zwV3CdEEXihcEEaJXP64waCpUEACgyDaV
REAxowhWQNovobhJHE/rxRk=
=w3xM
-----END PGP SIGNATURE-----

[Jim Fulton (jim-zope)]

On Oct 10, 2008, at 8:26 AM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Artur Zaprzała wrote:
>> As for your patch, looking at ZODB 3.8 I didn't find a possibility
>> for
>> self._reader to not exist or be None. Did I miss something?
>
> I think you are correct for the 3.8 branch: it alloacates the
> '_reader'
> in '__init__'. I was applying the fix to older versions, as well,
> where the '_reader' object is not created until the end of '_setDB',
> *after* the call to ''_resetCache'.

Does this mean the patch isn't needed for 3.8? It sounds like the test
would still be useful.

Jim

--
Jim Fulton
Zope Corporation

[Artur Zaprzała (arturz)]

The patch is required for 3.8, but without the "if getattr..." test, just the line:
self._reader._cache = cache

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> On Oct 10, 2008, at 8:26 AM, Tres Seaver wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Artur Zaprzała wrote:
>>> As for your patch, looking at ZODB 3.8 I didn't find a possibility
>>> for
>>> self._reader to not exist or be None. Did I miss something?
>> I think you are correct for the 3.8 branch: it alloacates the
>> '_reader'
>> in '__init__'. I was applying the fix to older versions, as well,
>> where the '_reader' object is not created until the end of '_setDB',
>> *after* the call to ''_resetCache'.
>
>
> Does this mean the patch isn't needed for 3.8? It sounds like the test
> would still be useful.

No, it means that the test for a non-existent '_reader' can be dropped
(I have done that in my 3,8 sandbox).

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI72x6+gerLs4ltQ4RAp1KAKC0HtkNw0fXNgoA+8cNOYcNuC+lFQCgyO7s
GNz/hZ/DtWGIR5E5kn6QQ+E=
=UD2f
-----END PGP SIGNATURE-----

[Jim Fulton (jim-zope)]

On Oct 10, 2008, at 10:53 AM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jim Fulton wrote:
>> On Oct 10, 2008, at 8:26 AM, Tres Seaver wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Artur Zaprzała wrote:
>>>> As for your patch, looking at ZODB 3.8 I didn't find a possibility
>>>> for
>>>> self._reader to not exist or be None. Did I miss something?
>>> I think you are correct for the 3.8 branch: it alloacates the
>>> '_reader'
>>> in '__init__'. I was applying the fix to older versions, as well,
>>> where the '_reader' object is not created until the end of '_setDB',
>>> *after* the call to ''_resetCache'.
>>
>>
>> Does this mean the patch isn't needed for 3.8? It sounds like the
>> test
>> would still be useful.
>
> No, it means that the test for a non-existent '_reader' can be dropped
> (I have done that in my 3,8 sandbox).

OK, well, whenever you're ready, go ahead and commit.

I assume that this is isolated enough that we don't need another 3.8.1
beta. I will review the diff after you commit to reassure myself that
this is true. :)

Jim

--
Jim Fulton
Zope Corporation

[Tres Seaver (tseaver)]

- ZODB 3.8 branch: http://svn.zope.org/ZODB/branches/3.8/?rev=91987&view=rev
  Will be available in 3.8.1.

[Tres Seaver (tseaver)]

Long since releaesd.