After the new rollout to cesium, buildds were marked as disabled by the buildd-manager with the following traceback.
2010-11-04 21:53:08+0000 [QueryProtocol,client] Scanning failed with: 'NoneType' object has no attribute 'processor'
2010-11-04 21:53:08+0000 [QueryProtocol,client] Traceback (most recent call last):
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/web/xmlrpc.py", line 318, in handleResponse
2010-11-04 21:53:08+0000 [QueryProtocol,client] self.factory.parseResponse(contents)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/web/xmlrpc.py", line 395, in parseResponse
2010-11-04 21:53:08+0000 [QueryProtocol,client] deferred.callback(response)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 318, in callback
2010-11-04 21:53:08+0000 [QueryProtocol,client] self._startRunCallbacks(result)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 424, in _startRunCallbacks
2010-11-04 21:53:08+0000 [QueryProtocol,client] self._runCallbacks()
2010-11-04 21:53:08+0000 [QueryProtocol,client] --- <exception caught here> ---
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 441, in _runCallbacks
2010-11-04 21:53:08+0000 [QueryProtocol,client] self.result = callback(self.result, *args, **kw)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/lib/lp/buildmaster/model/builder.py", line 298, in rescue_slave
2010-11-04 21:53:08+0000 [QueryProtocol,client] builder.verifySlaveBuildCookie(slave_build_id)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/lib/lp/buildmaster/model/builder.py", line 561, in verifySlaveBuildCookie
2010-11-04 21:53:08+0000 [QueryProtocol,client] slave_build_id)
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/lib/lp/buildmaster/model/buildfarmjobbehavior.py", line 69, in verifySlaveBuildCookie
2010-11-04 21:53:08+0000 [QueryProtocol,client] expected_cookie = self.buildfarmjob.generateSlaveBuildCookie()
2010-11-04 21:53:08+0000 [QueryProtocol,client] File "/srv/launchpad.net/codelines/soyuz-production-rev-11824/lib/lp/buildmaster/model/buildfarmjob.py", line 166, in generateSlaveBuildCookie
2010-11-04 21:53:08+0000 [QueryProtocol,client] if buildqueue.processor is None:
2010-11-04 21:53:08+0000 [QueryProtocol,client] exceptions.AttributeError: 'NoneType' object has no attribute 'processor'
2010-11-04 21:53:08+0000 [QueryProtocol,client]
2010-11-04 21:53:17+0000 [QueryProtocol,client] Builder adare failed a probe, count: 6
It looks like it's performing the slave build cookie check after a failure is detected and the current build is reset. But the Builder object is retained, so its cached BuildFarmJobBeh avior is used instead of IdleBuildBehavior.
I suspect that rescueIfLost shouldn't be called after a failure.