Example of a successful gate job that only failed on the last few steps in toci_quickstart.sh:
http://logs.openstack.org/48/518548/1/gate/legacy-tripleo-ci-centos-7-undercloud-oooq/da2d936/job-output.txt.gz#_2017-11-09_00_40_50_055344
2017-11-09 00:40:49.970367 | primary | +(./toci_quickstart.sh:109): du -L -ch /home/zuul/workspace/logs/ara_oooq /home/zuul/workspace/logs/delorean_logs /home/zuul/workspace/logs/devstack-gate-setup-host.txt /home/zuul/workspace/logs/devstack-gate-setup-workspace-new.txt /home/zuul/workspace/logs/full_logs.html /home/zuul/workspace/logs/postci.txt.gz /home/zuul/workspace/logs/quickstart_collect_logs.log /home/zuul/workspace/logs/quickstart_install.log /home/zuul/workspace/logs/README-logs.html /home/zuul/workspace/logs/reproduce.sh /home/zuul/workspace/logs/testrepository.subunit.gz /home/zuul/workspace/logs/undercloud 2017-11-09 00:40:49.970509 | primary | +(./toci_quickstart.sh:109): sort -rh 2017-11-09 00:40:49.970575 | primary | +(./toci_quickstart.sh:109): head -n 200 2017-11-09 00:40:50.055214 | primary | sort: write failed: standard output: Broken pipe 2017-11-09 00:40:50.055344 | primary | sort: write error
This is the problematic line: https://github.com/openstack-infra/tripleo-ci/blob/701ff27cfb11897a0c998ea6d3556fa94bfd5634/toci_quickstart.sh#L109
My suspicion is that the sorting happens in the /tmp directory which could get full (or has some other problem?) and the sort operation fails.
Possible solutions are:
* using `sort -T <dir>` and point it somewhere where there is more space. * reduce the amount of files that we check by using `du -d 1` for example
Example of a successful gate job that only failed on the last few steps in toci_quickstart.sh:
http:// logs.openstack. org/48/ 518548/ 1/gate/ legacy- tripleo- ci-centos- 7-undercloud- oooq/da2d936/ job-output. txt.gz# _2017-11- 09_00_40_ 50_055344
2017-11-09 00:40:49.970367 | primary | +(./toci_ quickstart. sh:109) : du -L -ch /home/zuul/ workspace/ logs/ara_ oooq /home/zuul/ workspace/ logs/delorean_ logs /home/zuul/ workspace/ logs/devstack- gate-setup- host.txt /home/zuul/ workspace/ logs/devstack- gate-setup- workspace- new.txt /home/zuul/ workspace/ logs/full_ logs.html /home/zuul/ workspace/ logs/postci. txt.gz /home/zuul/ workspace/ logs/quickstart _collect_ logs.log /home/zuul/ workspace/ logs/quickstart _install. log /home/zuul/ workspace/ logs/README- logs.html /home/zuul/ workspace/ logs/reproduce. sh /home/zuul/ workspace/ logs/testreposi tory.subunit. gz /home/zuul/ workspace/ logs/undercloud quickstart. sh:109) : sort -rh quickstart. sh:109) : head -n 200
2017-11-09 00:40:49.970509 | primary | +(./toci_
2017-11-09 00:40:49.970575 | primary | +(./toci_
2017-11-09 00:40:50.055214 | primary | sort: write failed: standard output: Broken pipe
2017-11-09 00:40:50.055344 | primary | sort: write error
This is the problematic line: https:/ /github. com/openstack- infra/tripleo- ci/blob/ 701ff27cfb11897 a0c998ea6d3556f a94bfd5634/ toci_quickstart .sh#L109
My suspicion is that the sorting happens in the /tmp directory which could get full (or has some other problem?) and the sort operation fails.
Possible solutions are:
* using `sort -T <dir>` and point it somewhere where there is more space.
* reduce the amount of files that we check by using `du -d 1` for example