HTTPRequest.resolve_url clears site thread global

[Martijn Pieters]

HTTPRequest.resolve_url clones the request to use a clean URL traversal environment, then when done, calls .close() on the cloned request. However, .close() emits an IEndRequestEvent event, which Five has registered a clearThreadSiteSubscriber handler for. This means that any and all persistent utility registrations in a nested site are no longer available for the rest of the (non-cloned) original request.

For example, ZCatalog uses request.resolve_url to find any objects that cannot be found through regular path traversal (see refreshCatalog, manage_catalogObject and reindexIndex for examples). If for any reason, there are stale entries in the catalog pointing to non-existing objects, ZCatalog will automatically also try to use resolve_url to find the object. A full reindex with stale objects thus leads to a reset of the site manager, and any subsequent reindexing of attributes that require access to persistent utilities fails.

The .resolve_url method should be altered to either not call .close() (a simple "del req" should suffice) or a new .cleanup() method should be added to do all cleanup work, which .resolve_url and .close then call, with only the IEndRequestEvent emit remaining in .close.

[Martijn Pieters (mjpieters)]

Patch against trunk; I'll see about backporting this to 2.12, 2.11 and 2.10 as it is quite critical to any large site that wants to do a full reindex to get rid of stale entries.

[Martijn Pieters (mjpieters)]

Patch committed in r104360:

  http://svn.zope.org/Zope/trunk/?rev=104360&view=rev

and backported to 2.12, 2.11 and 2.10:

  http://svn.zope.org/Zope/branches/2.12/?rev=104361&view=rev

  http://svn.zope.org/Zope/branches/2.11/?rev=104363&view=rev

  http://svn.zope.org/Zope/branches/2.10/?rev=104364&view=rev

[Andi Zeidler (witsch)]

the fix currently breaks subscribers to the `EndRequestEvent`: the following error occurs when a persistent registry tries to get unghosted so that such events can be dispatched:

  2009-10-15 10:58:50 ERROR ZServerPublisher exception caught
  Traceback (most recent call last):
    File ".../Zope2/src/ZServer/PubCore/ZServerPublisher.py", line 31, in __init__
      response=b)
    File ".../Zope2/src/ZPublisher/Publish.py", line 430, in publish_module
      environ, debug, request, response)
    File ".../Zope2/src/ZPublisher/Publish.py", line 256, in publish_module_standard
      if request is not None: request.close()
    File ".../Zope2/src/ZPublisher/BaseRequest.py", line 215, in close
      notify(EndRequestEvent(None, self))
    File ".../zope.event-3.4.1-py2.6.egg/zope/event/__init__.py", line 23, in notify
      subscriber(event)
    File ".../zope.component-3.7.1-py2.6.egg/zope/component/event.py", line 26, in dispatch
      for ignored in zope.component.subscribers(event, None):
    File ".../zope.component-3.7.1-py2.6.egg/zope/component/_api.py", line 138, in subscribers
      return sitemanager.subscribers(objects, interface)
    File ".../ZODB3-3.9.2-py2.6-macosx-10.6-i386.egg/ZODB/Connection.py", line 831, in setstate
      raise ConnectionStateError(msg)
  ConnectionStateError: Shouldn't load state for 0x0e22 when the connection is closed

the problem should be fixed by reversing the calls in "ZPublisher/BaseRequest.py", line 213, in close:

  def close(self):
      self.clear()
      notify(EndRequestEvent(None, self))

`self._held` is set to `None` in `clear()`, which causes the ZODB connection to be closed. however, event subscribers may still need to access it. triggering the event first should fix things again...

[Andi Zeidler (witsch)]

the above mentioned fix was committed in r105078:

  http://svn.zope.org/?rev=105078&view=rev

and ported to trunk, 2.11 and 2.10 in:

  http://svn.zope.org/?rev=105079&view=rev

  http://svn.zope.org/?rev=105080&view=rev

  http://svn.zope.org/?rev=105081&view=rev

i also tried to write a test for quite a while, but couldn't reproduce the error. sorry...