Lost builders don't have their jobs unassigned on rescue

[William Grant]

If a launchpad-buildd slave's idea of its current job doesn't match the master's, it is declared to be lost and BuilderInteractor.rescueIfLost attempts to abort the build and clean the slave up. But SlaveScanner.scan doesn't notice that a rescue is in progress, and continues to call updateBuild on the build that it knows to no longer be present on the builder. It should probably immediately reset the job so it can be dispatched to another builder while the lost slave is cleaned up.

This bug currently resolves in lost builders occasionally getting failed when they eventually transition to WAITING/ABORTED, because the status gets inappropriately passed down to a job that is in the wrong state. Affected builders can be recovered by simply reenabling them, as the cleanup has already taken place.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r16760 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/16760>.