Milestone:+index times out with enormous XRef query

Bug #1520281 reported by Colin Watson on 2015-11-26
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Critical
Colin Watson

Bug Description

OOPS-a6e1220cbc8ffb650971d176846a10b1 (beware, this OOPS is very large and will take a long time to load)

There's a gigantic XRef query in here taking around five seconds, from:

  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.server-3.8.6-py2.7.egg/zope/server/taskthreads.py", line 49, in handlerThread
    task.service()
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.server-3.8.6-py2.7.egg/zope/server/serverchannelbase.py", line 205, in service
    task.service()
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.server-3.8.6-py2.7.egg/zope/server/http/httptask.py", line 67, in service
    self.channel.server.executeRequest(self)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zc.zservertracelog-1.3.2-py2.7.egg/zc/zservertracelog/tracelog.py", line 124, in executeRequest
    response = self.application(env, start_response)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.app.wsgi-3.10.0-py2.7.egg/zope/app/wsgi/__init__.py", line 59, in __call__
    request = publish(request, handle_errors=handle_errors)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.publisher-3.12.6-py2.7.egg/zope/publisher/publish.py", line 132, in publish
    result = publication.callObject(request, obj)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/webapp/publication.py", line 463, in callObject
    return mapply(ob, request.getPositionalArguments(), request)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.publisher-3.12.6-py2.7.egg/zope/publisher/publish.py", line 107, in mapply
    return debug_call(obj, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.publisher-3.12.6-py2.7.egg/zope/publisher/publish.py", line 113, in debug_call
    return obj(*args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/webapp/publisher.py", line 443, in __call__
    return self.render()
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/webapp/publisher.py", line 428, in render
    return self.template()
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.browserpage-3.12.2-py2.7.egg/zope/browserpage/viewpagetemplatefile.py", line 83, in __call__
    return self.im_func(im_self, *args, **kw)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.browserpage-3.12.2-py2.7.egg/zope/browserpage/viewpagetemplatefile.py", line 51, in __call__
    sourceAnnotations=getattr(debug_flags, 'sourceAnnotations', 0),
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.pagetemplate-3.5.0_p1-py2.7.egg/zope/pagetemplate/pagetemplate.py", line 113, in pt_render
    strictinsert=0, sourceAnnotations=sourceAnnotations)()
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 271, in __call__
    self.interpret(self.program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 888, in do_useMacro
    self.interpret(macro)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 533, in do_optTag_tal
    self.do_optTag(stuff)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 518, in do_optTag
    return self.no_tag(start, program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 513, in no_tag
    self.interpret(program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 533, in do_optTag_tal
    self.do_optTag(stuff)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 522, in do_optTag
    self.interpret(program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 852, in do_condition
    self.interpret(block)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 533, in do_optTag_tal
    self.do_optTag(stuff)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 518, in do_optTag
    return self.no_tag(start, program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 513, in no_tag
    self.interpret(program)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 946, in do_defineSlot
    self.interpret(slot)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 583, in do_setLocal_tal
    self.engine.setLocal(name, self.engine.evaluateValue(expr))
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/tales.py", line 696, in evaluate
    return expression(self)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/expressions.py", line 217, in __call__
    return self._eval(econtext)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/expressions.py", line 194, in _eval
    ob = self._subexprs[-1](econtext)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/expressions.py", line 124, in _eval
    ob = self._traverser(ob, element, econtext)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.pagetemplate-3.5.0_p1-py2.7.egg/zope/pagetemplate/engine.py", line 66, in __call__
    request=request)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.traversing-3.14.0-py2.7.egg/zope/traversing/adapters.py", line 136, in traversePathElement
    return traversable.traverse(nm, further_path)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/zope.traversing-3.14.0-py2.7.egg/zope/traversing/adapters.py", line 42, in traverse
    attr = getattr(subject, name, _marker)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/propertycache.py", line 115, in __get__
    value = self.populate(instance)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/registry/browser/milestone.py", line 270, in bugtasks
    return [self._getListingItem(bugtask) for bugtask in self._bugtasks]
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/registry/browser/milestone.py", line 259, in _getListingItem
    badge_property = self._bug_badge_properties[bugtask]
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/propertycache.py", line 115, in __get__
    value = self.populate(instance)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/registry/browser/milestone.py", line 246, in _bug_badge_properties
    self._bugtasks)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/bugs/model/bugtask.py", line 1394, in getBugTaskBadgeProperties
    types=[u'specification']).keys())
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/xref/model.py", line 124, in findFromMany
    XRef.to_type.is_in(types) if types is not None else True))
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/store.py", line 997, in __iter__
    result = self._store._connection.execute(self._get_select())
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/databases/postgres.py", line 266, in execute
    return Connection.execute(self, statement, params, noresult)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/database.py", line 238, in execute
    raw_cursor = self.raw_execute(statement, params)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/databases/postgres.py", line 276, in raw_execute
    return Connection.raw_execute(self, statement, params)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/database.py", line 316, in raw_execute
    statement, params or ())
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/database.py", line 371, in _check_disconnect
    return function(*args, **kwargs)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/tracer.py", line 216, in trace
    attr(*args, **kwargs)
  File "/srv/launchpad.net/production/launchpad-rev-17850/lib/lp/services/webapp/adapter.py", line 697, in connection_raw_execute
    'SQL-%s' % connection._database.name, statement_to_log)
  File "/srv/launchpad.net/production/launchpad-rev-17850/eggs/timeline-0.0.3-py2.7.egg/timeline/timeline.py", line 91, in start
    result.backtrace = ''.join(self.format_stack())

Related branches

Colin Watson (cjwatson) on 2015-11-26
Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → In Progress
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
Colin Watson (cjwatson) on 2015-11-30
tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant) on 2015-12-01
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers