ZEO "extension methods" fail after ZEO restarts?

[Paul Winkler]

This is copied from a thread on zodb-dev:
http://mail.zope.org/pipermail/zodb-dev/2005-March/008582.html

The issue, with detailed repro instructions:
http://sourceforge.net/tracker/index.php?func=detail&aid=1106702&group_id=53818&atid=471702

Toby's response from dirstorage-users:
http://sourceforge.net/mailarchive/message.php?msg_id=10470537

Summary:

- if ZEO ever restarts, the DSToolkit Product can no
  longer do its job (forcing the storage into
  Snapshot mode) - instead ZEO raises
  DisconnectedError, even though all other Zope
  activity proceeds as normal, i.e. Zope is NOT
  disconnected from ZEO.
  Restarting Zope makes the symptom go away.

- I can reproduce this on Zope 2.7.3

- Toby can NOT reproduce this with Zope 2.6.x

- Toby says "DirectoryStorage is using
  ZEO's getExtensionMethods API,
  and I supect the problem will be there."

Now, I don't know squat about getExtensionMethods
except what I've learned browsing the ZEO source in
the past few minutes. But here's what I see:

From a brief look, DSToolkit is a very simple
Product. It contains a couple of methods that in turn
invoke ZEO extension methods. Here's an example:

    security.declareProtected('Manage DirectoryStorages','is_directorystorage')
    def is_directorystorage(self):
        if self._v_is_directory_storage is None:
            storage = self._p_jar._storage
            try:
                fn = storage.is_directory_storage
            except AttributeError:
                self._v_is_directory_storage = 0
            else:
                self._v_is_directory_storage = fn()
        return self._v_is_directory_storage

... which method of the storage is referred to in
BaseDirectoryStorage.py:

    def getExtensionMethods(self):
        return {'enter_snapshot': None,
                'leave_snapshot': None,
                'get_snapshot_code': None,
                'is_directory_storage': None,
                }

... which looks fine according to the API in ClientStorage.py; a bit further down in
BaseDirectoryStorage.py is the method definition:

    def is_directory_storage(self):
        return 1

So, everything in DirectoryStorage and DSToolkit
looks kosher to me; is this a ZEO bug?

[Tim Peters (tim-one)]

Googling shows that Toby added the ZODB+ZEO code for getExtensionMethods late 2002. AFAICT, he didn't check in docs or tests for it, just implementation code.

Since I don't know anything about it, and Zope Corp doesn't appear to make use of it, it's hard for me to justify spending work time it.

Best guess is that the problem is shallow, but likely shallower to Toby than to anyone else on the planet.

[Toby Dickenson (htrd)]

ClientStorage._handle_extensions sets up method proxy objects as attributes of the ClientStorage instance. Those proxy objects hold a reference to ClientStorage._server, so ClientStorage._handle_extensions needs to be called when ClientStorage._server changes valid. It gets called from notifyConnected which (thanks to deferred cache validation) is sometimes too soon.

The low impact change is to call ClientStorage._handle_extensions at more/different places. Maybe refactor the several assignments to self._server into their own method.

Im unlikely to get round to implementation/testing of this for a while.... can anyone else pick this up?

[Jim Fulton (jim-zope)]

And now there's an extension methods test. :)