IBuilder.updateBuild() doesn't deal with EINTR

[Julian Edwards]

In the new buildd manager, which now interleaves subprocesses via spawnProcess with other builder comms, we are getting "Interrupted system call" errors when polling builders. This is most likely because the reset script for another builder just finished and we get a SIGCHLD, which terminates the socket receive op.

{{{
2010-08-12 11:05:38+0100 [-] Disabling builder: http://yellow.buildd:8221/ -- (4
, 'Interrupted system call')
2010-08-12 11:05:38+0100 [-] Traceback (most recent call last):
2010-08-12 11:05:38+0100 [-] File "/srv/launchpad.net/codelines/soyuz-producti
on-rev-9648/lib/lp/buildmaster/model/builder.py", line 204, in updateBuilderStat
us
2010-08-12 11:05:38+0100 [-] builder.checkSlaveAlive()
2010-08-12 11:05:38+0100 [-] File "/srv/launchpad.net/codelines/soyuz-producti
on-rev-9648/lib/lp/buildmaster/model/builder.py", line 286, in checkSlaveAlive
2010-08-12 11:05:38+0100 [-] if self.slave.echo("Test")[0] != "Test":
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/xmlrpclib.py", line 1147
, in __call__
2010-08-12 11:05:38+0100 [-] return self.__send(self.__name, args)
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/xmlrpclib.py", line 1437, in __request
2010-08-12 11:05:38+0100 [-] verbose=self.__verbose
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/xmlrpclib.py", line 1185, in request
2010-08-12 11:05:38+0100 [-] errcode, errmsg, headers = h.getreply()
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/httplib.py", line 1199, in getreply
2010-08-12 11:05:38+0100 [-] response = self._conn.getresponse()
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/httplib.py", line 928, in getresponse
2010-08-12 11:05:38+0100 [-] response.begin()
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/httplib.py", line 385, in begin
2010-08-12 11:05:38+0100 [-] version, status, reason = self._read_status()
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/httplib.py", line 343, in _read_status
2010-08-12 11:05:38+0100 [-] line = self.fp.readline()
2010-08-12 11:05:38+0100 [-] File "/usr/lib/python2.5/socket.py", line 331, in readline
2010-08-12 11:05:38+0100 [-] data = recv(1)
2010-08-12 11:05:38+0100 [-] error: (4, 'Interrupted system call')
2010-08-12 11:05:39+0100 [-] yellow was made unavailable, resetting attached job
2010-08-12 11:05:40+0100 [-] Dispatching: <lawrencium:http://lawrencium.ppa:8221/>
}}}

[Francis J. Lacoste (flacoste)]

Escaling to Critical as this is killing time-sensitive OEM builds.

[Julian Edwards (julian-edwards)]

A fresh cowboy this morning seems to have cleaned up the problem. I'll land
the fix properly now.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r11345 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/11345>.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r11348 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/11348>.