Impossible to determine how a utility was registered

[Wichert Akkerman]

GenericSetup provides a way in Zope2 to manage local utilities. At the moment it is impossible to do this reliably since it can not detect if the current registration is similar to requested registration details, and just re-registering utilities will lead to loss of data in persistent utilities.

In order to support this cleanly there should be a query interface that allows one to get the registration information from the CA. As far as I could tell at the moment one can only do a query, which, for example, makes it impossible to determine if a factory was registered.

[Philipp von Weitershausen (philikon)]

Wichert Akkerman wrote:
> GenericSetup provides a way in Zope2 to manage local utilities. At the
> moment it is impossible to do this reliably since it can not detect if
> the current registration is similar to requested registration details,
> and just re-registering utilities will lead to loss of data in
> persistent utilities.

I don't understand why there would be data loss. You don't have to
reinstantiate an object just to re-register it (see below). You *do*
probably want to un-register first before re-registering in case
registration details changed (e.g. the name or provided interface).

> In order to support this cleanly there should be a query interface that
> allows one to get the registration information from the CA.

There is. Every component registry (aka site manager) supports the
following methods from zope.component.interfaces.IComponentRegistry:

* registeredUtilities()
* registeredAdapters()
* registeredSubscriptionAdapters()
* registeredHandlers()

They each return an iterable of zope.component.interfaces.IRegistration
objects which contain all the necessary information about how each
component was registered. Specific interfaces like IUtilityRegistration
tell you how to talk to those objects. E.g.:

   sm = cmf_site.getSiteManager()
   for reg in sm.registeredUtilities():
       print "%r was registered for %r as %r" (reg.component,
reg.provided, reg.name)

> As far as I could tell at the moment one can only do a query, which, for example,
> makes it impossible to determine if a factory was registered.

Utilities are *never* registered by factory. Sure, the <utility /> ZCML
directive takes a 'factory' argument, but if you use that instead of the
'component' argument, the directive handler will simply call the factory
and obtain the utility object, which is then registered.

Local persistent utilities are *always* registered as instances. In
other words, there's a persistent object somewhere that you then
register with the component registry.

[Martijn Pieters (mjpieters)]

On 17. juni. 2008, at 13.05, Philipp von Weitershausen wrote:
> I don't understand why there would be data loss. You don't have to
> reinstantiate an object just to re-register it (see below). You *do*
> probably want to un-register first before re-registering in case
> registration details changed (e.g. the name or provided interface).

But the GenericSetup components step does just that, see https://bugs.launchpad.net/zope-cmf/+bug/240170
. I do think that there is enough info available there to determine if
the registration doesn't need to be re-run though. I think Wichert is
chasing a red herring here, personally ;-)

> There is. Every component registry (aka site manager) supports the
> following methods from zope.component.interfaces.IComponentRegistry:
>
> * registeredUtilities()
> * registeredAdapters()
> * registeredSubscriptionAdapters()
> * registeredHandlers()
>
> They each return an iterable of
> zope.component.interfaces.IRegistration
> objects which contain all the necessary information about how each
> component was registered. Specific interfaces like
> IUtilityRegistration
> tell you how to talk to those objects. E.g.:
>
> sm = cmf_site.getSiteManager()
> for reg in sm.registeredUtilities():
> print "%r was registered for %r as %r" (reg.component,
> reg.provided, reg.name)

Wouldn't it be a tad ineffecient to have to loop over all
registrations to find information for one or at most a few utilities?
Not that we need this info, see below.

> Utilities are *never* registered by factory. Sure, the <utility />
> ZCML
> directive takes a 'factory' argument, but if you use that instead of
> the
> 'component' argument, the directive handler will simply call the
> factory
> and obtain the utility object, which is then registered.
>
> Local persistent utilities are *always* registered as instances. In
> other words, there's a persistent object somewhere that you then
> register with the component registry.

Indeed, I was already wondering why Wichert had raised this bug. :-)

Martijn

[Philipp von Weitershausen (philikon)]

Martijn Pieters wrote:
> On 17. juni. 2008, at 13.05, Philipp von Weitershausen wrote:
>> I don't understand why there would be data loss. You don't have to
>> reinstantiate an object just to re-register it (see below). You *do*
>> probably want to un-register first before re-registering in case
>> registration details changed (e.g. the name or provided interface).
>
> But the GenericSetup components step does just that, see https://bugs.launchpad.net/zope-cmf/+bug/240170
> . I do think that there is enough info available there to determine if
> the registration doesn't need to be re-run though. I think Wichert is
> chasing a red herring here, personally ;-)

I gather from this bug report of yours that GenericSetup does NOT do
this yet but you'd like it to... Surely sm.registeredUtilities() equips
GenericSetup with enough information to check whether a new registration
must be made, an existing must be modified (IOW, unregister +
reregister) or nothing has changed.

So, can we close this bug then?

> Wouldn't it be a tad ineffecient to have to loop over all
> registrations to find information for one or at most a few utilities?

registeredUtilities() only loops over the registrations made with *this
particular* site manager. How many local tools does a CMF site have? 20?
30? So really not that many.

Plus, you're going to have to loop over the data structures *somewhere*
to filter out the ones you like, so I don't really see a way to
magically optimize this (unless the internal data structures of the
component registry was changed). But I call premature optimization on
this one :) It's not like you need to do this with every request, this
just needs to be done once during a GenericSetup step.

[Martijn Pieters (mjpieters)]

On 17. juni. 2008, at 14.11, Philipp von Weitershausen wrote:
> So, can we close this bug then?

*I* think it can be closed, but I am not the reporter ;-)

>> Wouldn't it be a tad ineffecient to have to loop over all
>> registrations to find information for one or at most a few utilities?
>
> registeredUtilities() only loops over the registrations made with
> *this
> particular* site manager. How many local tools does a CMF site have?
> 20?
> 30? So really not that many.

Of course.

> Plus, you're going to have to loop over the data structures
> *somewhere*
> to filter out the ones you like, so I don't really see a way to
> magically optimize this (unless the internal data structures of the
> component registry was changed). But I call premature optimization on
> this one :) It's not like you need to do this with every request, this
> just needs to be done once during a GenericSetup step.

I was thinking along the lines of queryUtility here..

Martijn

[Wichert Akkerman (wichert)]

I see no mention of IRegistration anywhere. zope.component.interfaces says:

    def registeredUtilities():
        """Return an iterable of IUtilityRegistrations.

        These registrations describe the current utility registrations
        in the object.
        """

However a recursive grep through all of zope.* reveals that IUtilityRegistrations does not exist anywhere. The same holds for IAdapterRegistrations, ISubscriptionAdapterRegistrations and IHandlerRegistrations.

Looking more.. I see that this is a very unfortunate typography problem: what is returned is an iterable of IUtilityRegistration (singular) instances. The plural form sounded perfectly sensible to me, which has led me into a dead end twice now. Gah.

[Wichert Akkerman (wichert)]

Martijn makes a good point: site managers already have all the logic to find the right registration and look at parent managers. A set of query*Registration methods parallel to the existing query* methods would be ideal and remove a lot of code that would have to be duplicated.

[Martijn Pieters (mjpieters)]

On 17. juni. 2008, at 14.57, Wichert Akkerman wrote:
> Martijn makes a good point: site managers already have all the logic
> to
> find the right registration and look at parent managers. A set of
> query*Registration methods parallel to the existing query* methods
> would
> be ideal and remove a lot of code that would have to be duplicated.

But why do you need the registration info? You have the interface, all
you need is an isinstance check or an 'is' test to see if the returned
utility is the same as what components.xml wants to register,
depending on wether or not it's a factory or a component registration.

Martijn

[Martijn Pieters (mjpieters)]

On 17. juni. 2008, at 14.39, Martijn Pieters wrote:
> On 17. juni. 2008, at 14.11, Philipp von Weitershausen wrote:
>> So, can we close this bug then?
>
> *I* think it can be closed, but I am not the reporter ;-)

A quick chat with Hanno here made it clear this bug is not only about
my GenericSetup components.xml clobbering existing persistent
utilities bug. There is also a problem of exporting components, at
which time you'd need to know if a factory was used.

Martijn

[Philipp von Weitershausen (philikon)]

Wichert Akkerman wrote:
> I see no mention of IRegistration anywhere.

It's just the base interface for IUtilityRegistration. As I've tried to
spell out in my first reply to this bug, all of those interfaces are in
zope.component.interfaces.

> zope.component.interfaces says:
>
> def registeredUtilities():
> """Return an iterable of IUtilityRegistrations.
>
> These registrations describe the current utility registrations
> in the object.
> """
>
> However a recursive grep through all of zope.* reveals that
> IUtilityRegistrations does not exist anywhere. The same holds for
> IAdapterRegistrations, ISubscriptionAdapterRegistrations and
> IHandlerRegistrations.
>
> Looking more.. I see that this is a very unfortunate typography problem:
> what is returned is an iterable of IUtilityRegistration (singular)
> instances. The plural form sounded perfectly sensible to me, which has
> led me into a dead end twice now. Gah.

Is it too far-fetched to grep for the singular when the plural doesn't
produce any results? Actually, what I did was search in the very same
interface module first instead of grepping through all of Zope. A good
editor will do progressive search as you type and you'll end up with the
right match before you stopped typing :)

> Martijn makes a good point: site managers already have all the logic to
> find the right registration and look at parent managers. A set of
> query*Registration methods parallel to the existing query* methods would
> be ideal and remove a lot of code that would have to be duplicated.

I'm not so sure you want those inspection methods to involve sites
higher up the hierarchy. They're mostly there to perform component
management like GenericSetup does.

Also, when looking at the code, you'll notice that the component
registry (aka site manager) keeps a special data structure just for
those registered*() methods. Why? Because the way you register things is
not necessarily the way they are put into the actual lookup structures
of component registry. So even if you added such methods they probably
couldn't work on the lookup datastructures but would simply wrappers
around registered*().

That said, if you'd like to introduce such methods, why not. You could
do it on five.localsitemanager to begin with and add the feature to
zope.component trunk so that it'd be available in next Zope version.

[Wichert Akkerman (wichert)]

Aside from the convenience methods to iterate over parent site managers there is the real issue of missing information about factories. For tools like GenericSetup that need to round-trip utility registration information that is critical.

Unfortuantely while the utility creation methods such as the zcml directory know about factories the component registry itself has no such concept. The directive just calls the factory registers the resulting instance as the component providing the utility.

I will investigate if there is a convenient way to add that information to the component registry. If that is not possible GenericSetup will need to keep its own utility registry, something I'ld rather avoid.

[Christian Zagrodnick (zagy)]

On 23.07.2008, at 10:43, Wichert Akkerman wrote:

> Aside from the convenience methods to iterate over parent site
> managers
> there is the real issue of missing information about factories. For
> tools like GenericSetup that need to round-trip utility registration
> information that is critical.

Note that it is not necessaryly possible to recreate a utility because
it may store data. This is for local utilities only of course. But
global utilities are re-created on startup anyway.

--
Christian Zagrodnick · <email address hidden>
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel [phone] · fax [phone]
Zope and Plone consulting and development

[Wichert Akkerman (wichert)]

I implemented a fix for this on wichert-utility-factories branch of zope.component.

[Wichert Akkerman (wichert)]

Merged to trunk.

[Christian Zagrodnick (zagy)]

was released with zope.component 3.5