Intermittent failure on a-b* : FATAL Unable to delete script file, IOException2 error

[Deepti B. Kalakeri]

Intermittent failure on a-b*, https://android-build.linaro.org/jenkins/job/linaro-android_panda-ics-gcc47-tilt-tracking-blob/103/console job failed with the FATAL Unable to delete script file, IOException2 error.

Here is the more details on the error:
FATAL: Unable to delete script file /tmp/hudson5081376341758221407.sh
hudson.util.IOException2: remote file operation failed: /tmp/hudson5081376341758221407.sh at hudson.remoting.Channel@12fa0d6:i-14a6976d
 at hudson.FilePath.act(FilePath.java:828)
 at hudson.FilePath.act(FilePath.java:814)
 at hudson.FilePath.delete(FilePath.java:1119)
 at hudson.plugins.shell.ShellWithStatus.perform(ShellWithStatus.java:275)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
 at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:703)
 at hudson.model.Build$RunnerImpl.build(Build.java:178)
 at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
 at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:473)
 at hudson.model.Run.run(Run.java:1410)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 at hudson.model.ResourceController.execute(ResourceController.java:88)
 at hudson.model.Executor.run(Executor.java:238)
Caused by: hudson.remoting.ChannelClosedException: channel is already closed
 at hudson.remoting.Channel.send(Channel.java:499)
 at hudson.remoting.Request.call(Request.java:110)
 at hudson.remoting.Channel.call(Channel.java:681)
 at hudson.FilePath.act(FilePath.java:821)
 ... 13 more
Caused by: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
 at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2570)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
 at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.Request.call(Request.java:149)
 at hudson.remoting.Channel.call(Channel.java:681)
 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
 at $Proxy56.join(Unknown Source)
 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:859)
 at hudson.Launcher$ProcStarter.join(Launcher.java:345)
 at hudson.plugins.shell.ShellWithStatus.perform(ShellWithStatus.java:246)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
 at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:703)
 at hudson.model.Build$RunnerImpl.build(Build.java:178)
 at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
 at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:473)
 at hudson.model.Run.run(Run.java:1410)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 at hudson.model.ResourceController.execute(ResourceController.java:88)
 at hudson.model.Executor.run(Executor.java:238)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.Request.abort(Request.java:273)
 at hudson.remoting.Channel.terminate(Channel.java:732)
 at hudson.remoting.Channel$ReaderThread.run(Channel.java:1157)
Caused by: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
 at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2570)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
 at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)

I got reference to open bugs on the same, here are few of them, though one of them is for windows slaves:

https://issues.jenkins-ci.org/browse/JENKINS-12235
https://issues.jenkins-ci.org/browse/JENKINS-11097

There are opinions that the error are seen because the slaves are Out Of Memory. I did not get access to the slave which ran this job, when the error occurred.

[Deepti B. Kalakeri (deeptik)]

Looks like we the error seem to dis appear with the build https://android-build.linaro.org/jenkins/job/linaro-android_panda-ics-gcc47-tilt-tracking-blob/105/, that was started by Paul.

Will need to wait and see if we get the errors again.

[Paul Sokolovsky (pfalcon)]

The stacktrace from the linked build log is how instance being terminated while build is still running looks like. And likely cause of instance termination is that it hanged. So, this behavior is "expected", the main problem is in lp:940226, and this bug can be closed.