devstack-gate-failure-due-to-screen-n-dhcp-log-not-found

[Devdatta Kulkarni]

https://review.openstack.org/#/c/298342/3

http://logs.openstack.org/42/298342/3/check/gate-solum-devstack-dsvm/89901c0/console.html#_2016-03-28_18_52_35_561

Stack trace:

2016-03-28 18:52:35.516 | [SCP] uploading file: '/srv/static/logs/42/298342/3/check/gate-solum-devstack-dsvm/89901c0/logs/screen-n-dhcp.txt'
2016-03-28 18:52:35.561 | ERROR: Failed to upload files
2016-03-28 18:52:35.561 | 4: java.io.IOException: java.io.FileNotFoundException: /home/jenkins/workspace/gate-solum-devstack-dsvm/logs/screen-n-dhcp.txt (No such file or directory)
2016-03-28 18:52:35.561 | at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:689)
2016-03-28 18:52:35.562 | at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:540)
2016-03-28 18:52:35.562 | at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:492)
2016-03-28 18:52:35.562 | at be.certipost.hudson.plugin.SCPSite.upload(SCPSite.java:271)
2016-03-28 18:52:35.562 | at be.certipost.hudson.plugin.SCPRepositoryPublisher.perform(SCPRepositoryPublisher.java:250)
2016-03-28 18:52:35.562 | at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
2016-03-28 18:52:35.562 | at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
2016-03-28 18:52:35.562 | at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
2016-03-28 18:52:35.562 | at hudson.model.Build$BuildExecution.post2(Build.java:185)
2016-03-28 18:52:35.562 | at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
2016-03-28 18:52:35.562 | at hudson.model.Run.execute(Run.java:1763)
2016-03-28 18:52:35.563 | at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
2016-03-28 18:52:35.563 | at hudson.model.ResourceController.execute(ResourceController.java:98)
2016-03-28 18:52:35.563 | at hudson.model.Executor.run(Executor.java:410)
2016-03-28 18:52:35.563 | Caused by: java.io.IOException: java.io.FileNotFoundException: /home/jenkins/workspace/gate-solum-devstack-dsvm/logs/screen-n-dhcp.txt (No such file or directory)
2016-03-28 18:52:35.563 | at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:169)
2016-03-28 18:52:35.563 | at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:636)
2016-03-28 18:52:35.563 | ... 13 more
2016-03-28 18:52:35.563 | Caused by: java.io.FileNotFoundException: /home/jenkins/workspace/gate-solum-devstack-dsvm/logs/screen-n-dhcp.txt (No such file or directory)
2016-03-28 18:52:35.563 | at java.io.FileInputStream.open(Native Method)
2016-03-28 18:52:35.564 | at java.io.FileInputStream.<init>(FileInputStream.java:146)
2016-03-28 18:52:35.564 | at hudson.FilePath$33.invoke(FilePath.java:1761)
2016-03-28 18:52:35.564 | at hudson.FilePath$33.invoke(FilePath.java:1754)
2016-03-28 18:52:35.564 | at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
2016-03-28 18:52:35.564 | at hudson.remoting.UserRequest.perform(UserRequest.java:120)
2016-03-28 18:52:35.564 | at hudson.remoting.UserRequest.perform(UserRequest.java:48)
2016-03-28 18:52:35.564 | at hudson.remoting.Request$2.run(Request.java:326)
2016-03-28 18:52:35.564 | at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
2016-03-28 18:52:35.564 | at java.util.concurrent.FutureTask.run(FutureTask.java:262)
2016-03-28 18:52:35.564 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2016-03-28 18:52:35.565 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2016-03-28 18:52:35.565 | at java.lang.Thread.run(Thread.java:745)

[Devdatta Kulkarni (devdatta-kulkarni)]

The issue seems to be that the screen-n-dhcp.txt file is not created and the Jenkins plugin code that uploads all the log files to logs.openstack.org fails.

Most possible cause of this issue might be that the plugin code started doing strict checking for file existence since earlier too there used to be no screen-n-dhcp.txt generated but it was not a problem. For instance, check the following logs from a merged patch

http://logs.openstack.org/40/293640/6/check/gate-solum-devstack-dsvm/dd0b1f5/logs/

(merged) https://review.openstack.org/#/c/293640/

If you look, screen-n-dhcp.txt is empty file. But this seemed to be not an issue then.

Therefore the conclusion that something changed in the Jenkins plugin code which is causing the gate failure.

As a workaround, following approaches were tried:
- enabling n-dhcp service via settings file in the Solum devstack plugin. The idea here was that if we enable n-dhcp, may be it will generate the corresponding log file. But this did not work.
- Create a dummy screen-n-dhcp file with some dummy content in the post_test_hook. This seem to work as seen in the following patch: https://review.openstack.org/#/c/298342/5

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/298342
Committed: https://git.openstack.org/cgit/openstack/solum/commit/?id=00853b04c6e4460d363618f19bb039ebc3f0cf8c
Submitter: Jenkins
Branch: master

commit 00853b04c6e4460d363618f19bb039ebc3f0cf8c
Author: Devdatta Kulkarni <email address hidden>
Date: Mon Mar 28 12:24:18 2016 -0500

    Unblocking the dsvm gate

    - Creating temp n-dhcp log file
    - Commenting out installation of Docker as we don't use
      Docker on the gate

    Partial-Bug: #1563019

    Change-Id: Idc18bc8e28e15777d544b9537dde76a329aba329