gate error: Looks like the node went offline during the build. Check the slave log for the details.FATAL: /var/lib/jenkins/logs/slaves/*/slave.log (No such file or directory)

[Joe Gordon]

Seen in the gate

Looks like the node went offline during the build. Check the slave log for the details.FATAL: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-1572924/slave.log (No such file or directory)
java.io.FileNotFoundException: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-1572924/slave.log (No such file or directory)
  at java.io.RandomAccessFile.open(Native Method)
  at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
  at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:397)
  at org.kohsuke.stapler.framework.io.LargeText$2.open(LargeText.java:120)
  at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:210)
  at hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:159)
  at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:579)
  at hudson.model.Run.execute(Run.java:1665)
  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
  at hudson.model.ResourceController.execute(ResourceController.java:88)
  at hudson.model.Executor.run(Executor.java:246)

This failure is making it all the way to gerrit (https://review.openstack.org/#/c/75918/)

http://logs.openstack.org/18/75918/1/check/gate-nova-python27/9c26936/console.html

message:" Looks like the node went offline during the build" AND message:"slave.log \(No such file or directory\)" AND filename:"console.html"

[Joe Gordon (jogo)]

23:30 < jeblair> jog0, zaro: the incidences of this in the jenkins log strongly
                 correlate with the gearman plugin de-registering jobs
23:30 < jeblair> jog0, zaro: it's possible it's more complicated than that, but it's
                 worth a look.
23:31 < jeblair> jog0, zaro: http://paste.openstack.org/show/69188/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/76043

[Daneyon Hansen (danehans)]

I have just been affected by this bug. Here is a snippet from the jenkins logs:

2014-02-25 19:55:28.656 | Finished in 14.72 seconds
2014-02-25 19:55:28.656 | 240 examples, 0 failures
2014-02-25 19:55:28.907 | Looks like the node went offline during the build. Check the slave log for the details.FATAL: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-1649626/slave.log (No such file or directory)
2014-02-25 19:55:28.907 | java.io.FileNotFoundException: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-1649626/slave.log (No such file or directory)
2014-02-25 19:55:28.907 | at java.io.RandomAccessFile.open(Native Method)
2014-02-25 19:55:28.907 | at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
2014-02-25 19:55:28.907 | at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:397)
2014-02-25 19:55:28.908 | at org.kohsuke.stapler.framework.io.LargeText$2.open(LargeText.java:120)
2014-02-25 19:55:28.908 | at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:210)
2014-02-25 19:55:28.908 | at hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:159)
2014-02-25 19:55:28.908 | at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:579)
2014-02-25 19:55:28.908 | at hudson.model.Run.execute(Run.java:1665)
2014-02-25 19:55:28.908 | at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
2014-02-25 19:55:28.908 | at hudson.model.ResourceController.execute(ResourceController.java:88)
2014-02-25 19:55:28.908 | at hudson.model.Executor.run(Executor.java:246)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/76043
Committed: https://git.openstack.org/cgit/openstack-infra/elastic-recheck/commit/?id=0abf712fb4dc693ed318f051978549a3142ef9b7
Submitter: Jenkins
Branch: master

commit 0abf712fb4dc693ed318f051978549a3142ef9b7
Author: Joe Gordon <email address hidden>
Date: Mon Feb 24 15:37:04 2014 -0800

    Add fingerprint for bug 1284371

    bug 1284371 is an infra bug.

    Change-Id: Id000cb37192c67616e208ba81f0c7e1fc39e534e
    Related-Bug: #1284371

[davidlenwell (dlenwell)]

I am having this same issue on .. https://review.openstack.org/#/c/78683/

 Looks like the node went offline during the build. Check the slave log for the details.FATAL: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-2246223/slave.log (No such file or directory)
2014-03-06 18:06:59.227 | java.io.FileNotFoundException: /var/lib/jenkins/logs/slaves/bare-precise-hpcloud-az2-2246223/slave.log (No such file or directory)
2014-03-06 18:06:59.227 | at java.io.RandomAccessFile.open(Native Method)
2014-03-06 18:06:59.227 | at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
2014-03-06 18:06:59.227 | at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:397)
2014-03-06 18:06:59.227 | at org.kohsuke.stapler.framework.io.LargeText$2.open(LargeText.java:120)
2014-03-06 18:06:59.227 | at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:210)
2014-03-06 18:06:59.227 | at hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:159)
2014-03-06 18:06:59.227 | at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:579)
2014-03-06 18:06:59.227 | at hudson.model.Run.execute(Run.java:1665)
2014-03-06 18:06:59.227 | at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
2014-03-06 18:06:59.228 | at hudson.model.ResourceController.execute(ResourceController.java:88)
2014-03-06 18:06:59.228 | at hudson.model.Executor.run(Executor.java:246)

[Tom Fifield (fifieldt)]

same issue on https://review.openstack.org/#/c/84866/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/309724

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/309724
Committed: https://git.openstack.org/cgit/openstack-infra/elastic-recheck/commit/?id=15eb9df23024e7012a988b84c219d5fba33df770
Submitter: Jenkins
Branch: master

commit 15eb9df23024e7012a988b84c219d5fba33df770
Author: Matt Riedemann <email address hidden>
Date: Sat Apr 23 17:51:44 2016 -0400

    Update fingerprint for bug 1284371

    There is a new message for this bug so add that to the
    existing fingerprint. This started spiking on 4/20.

    81 hits in voting jobs, check and gate, all branches.

    Change-Id: Ic7915078064bd1bad50e7f27721f3f202dab9691
    Related-Bug: #1284371

[Clark Boylan (cboylan)]

This was fixed by stopping our use of Jenkins.