[staging] Looks like some jenkins BVT slave node works incorrectly

[Timur Nurlygayanov]

We can see some strange errors which eventually appear during the BVT tests execution, looks like we have some broken Jenkins slave node.

Failed jobs:
http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.centos.bvt_1/60/console
http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.ubuntu.bvt_2/58/console
http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.ubuntu.bvt_2/64/console

Traceback:
FATAL: Unable to delete script file /tmp/hudson6852210277874090469.sh
hudson.util.IOException2: remote file operation failed: /tmp/hudson6852210277874090469.sh at hudson.remoting.Channel@35089bab:mc2n5-srt.srt.mirantis.net
 at hudson.FilePath.act(FilePath.java:901)
 at hudson.FilePath.act(FilePath.java:878)
 at hudson.FilePath.delete(FilePath.java:1263)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:101)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
 at hudson.model.Build$BuildExecution.build(Build.java:199)
 at hudson.model.Build$BuildExecution.doRun(Build.java:160)
 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
 at hudson.model.Run.execute(Run.java:1593)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 at hudson.model.ResourceController.execute(ResourceController.java:88)
 at hudson.model.Executor.run(Executor.java:247)
Caused by: hudson.remoting.ChannelClosedException: channel is already closed
 at hudson.remoting.Channel.send(Channel.java:524)
 at hudson.remoting.Request.call(Request.java:129)
 at hudson.remoting.Channel.call(Channel.java:722)
 at hudson.FilePath.act(FilePath.java:894)
 ... 13 more
Caused by: java.net.SocketException: Socket closed
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.read(SocketInputStream.java:146)
 at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
 at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
 at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265)
 at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558)
 at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
 at hudson.remoting.Command.readFrom(Command.java:92)
 at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
FATAL: hudson.remoting.RequestAbortedException: java.net.SocketException: Socket closed
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.net.SocketException: Socket closed
 at hudson.remoting.Request.call(Request.java:174)
 at hudson.remoting.Channel.call(Channel.java:722)
 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:162)
 at $Proxy51.join(Unknown Source)
 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:925)
 at hudson.Launcher$ProcStarter.join(Launcher.java:360)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
 at hudson.model.Build$BuildExecution.build(Build.java:199)
 at hudson.model.Build$BuildExecution.doRun(Build.java:160)
 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
 at hudson.model.Run.execute(Run.java:1593)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 at hudson.model.ResourceController.execute(ResourceController.java:88)
 at hudson.model.Executor.run(Executor.java:247)
Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Socket closed
 at hudson.remoting.Request.abort(Request.java:299)
 at hudson.remoting.Channel.terminate(Channel.java:782)
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.net.SocketException: Socket closed
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.read(SocketInputStream.java:146)
 at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
 at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
 at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2265)
 at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2558)
 at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2568)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
 at hudson.remoting.Command.readFrom(Command.java:92)
 at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

[Igor Belikov (ibelikov)]

http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.ubuntu.bvt_2/64/console and http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.centos.bvt_1/60/console are results of power maintenance in srt on jan 24th, it affected jenkins-product.

http://jenkins-product.srt.mirantis.net:8080/view/Staging%20Mirrors/job/6.1.staging.ubuntu.bvt_2/58/console is a short term networking issue. The node itself is working fine and I don't see and I don't see any other recent occurrences of the same problem. This should be investigated if the problem reappears.

[Timur Nurlygayanov (tnurlygayanov)]

So, the issue is not invalid, but fixed. We had the real issue and we have fixed it.