API timeouts broken and returning no useful data...

[Robert Collins]

       OOPS-1907C2337, OOPS-1907D1360, OOPS-1907D687, OOPS-1907F790, OOPS-1907G1375
       OOPS-1907A567, OOPS-1907B612, OOPS-1907C486, OOPS-1907G2210, OOPS-1907G542

these oops have tiny (1-10ms) durations, and claim to have taken no external actions at all, but the exceptions (e.g. in OOPS-1907G542) are clearly of db calls that have timed out.

The most likely cause I can think of is something starting a new publication cycle for the error (this happens in web requests too - but only *after* errorlog.raising() is called to capture the data).

This has crippled our data gathering, marking critical as it blocks our ability to fix other critical bugs.

[William Grant (wgrant)]

The recent lazr.restful exception changes broke this. The exception handler in lazr.restful._resource gets the view for the exception, somewhat like this:

Traceback (most recent call last):
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/publish.py", line 134, in publish
    result = publication.callObject(request, obj)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/lazr.restful-0.18.1-py2.6.egg/lazr/restful/publisher.py", line 211, in callObject
    WebServicePublicationMixin, self).callObject(request, object)
  File "/home/wgrant/launchpad/lp-branches/devel/lib/canonical/launchpad/webapp/publication.py", line 488, in callObject
    return mapply(ob, request.getPositionalArguments(), request)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/publish.py", line 109, in mapply
    return debug_call(obj, args)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/publish.py", line 115, in debug_call
    return obj(*args)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/lazr.restful-0.18.1-py2.6.egg/lazr/restful/_resource.py", line 923, in __call__
    view = getMultiAdapter((e, self.request), name="index.html")
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.component-3.9.3-py2.6.egg/zope/component/_api.py", line 109, in getMultiAdapter
    adapter = queryMultiAdapter(objects, interface, name, context=context)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.component-3.9.3-py2.6.egg/zope/component/_api.py", line 122, in queryMultiAdapter
    return sitemanager.queryMultiAdapter(objects, interface, name, default)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.component-3.9.3-py2.6.egg/zope/component/registry.py", line 240, in queryMultiAdapter
    objects, interface, name, default)
  File "/home/wgrant/launchpad/lp-sourcedeps/eggs/zope.interface-3.5.2-py2.6-linux-x86_64.egg/zope/interface/adapter.py", line 526, in queryMultiAdapter
    result = factory(*objects)
  File "/home/wgrant/launchpad/lp-branches/devel/lib/canonical/launchpad/webapp/error.py", line 246, in __init__
    clear_request_started()
  File "/home/wgrant/launchpad/lp-branches/devel/lib/canonical/launchpad/webapp/adapter.py", line 205, in clear_request_started
    raise Exception("I should not be called here!")

So it instantiates the error handling view, which resets the request in __init__ so the page can actually do stuff.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r12876 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12876>.