qemu: tar: Unexpected EOF in archive in copyup()

[Diogo Matsubara]

Sometimes MAAS adt tests fail to copy artifacts from the testbed to the host.

adt-run1: testbed executing test finished with exit status 0
adt-run2: >> copyup /tmp/adt-run.wP0YqQ/ubtree0t-maas-package-test-stdout /home/ubuntu/jenkins-jobs/workspace/utopic-adt-maas/results/ubtree0t-maas-package-test-stdout
adt-run2: << ok
adt-run2: >> copyup /tmp/adt-run.wP0YqQ/ubtree0t-maas-package-test-stderr /home/ubuntu/jenkins-jobs/workspace/utopic-adt-maas/results/ubtree0t-maas-package-test-stderr
adt-run2: << ok
adt-run: & ubtree0t-maas-package-test: - - - - - - - - - - results - - - - - - - - - -
ubtree0t-maas-package-test PASS
adt-run2: >> copyup /tmp/adt-run.wP0YqQ/ubtree0t-maas-package-test-artifacts/ /home/ubuntu/jenkins-jobs/workspace/utopic-adt-maas/results/artifacts/
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
qemu: terminating on signal 15 from pid 353
<VirtSubproc>: failure: copyup destination failed, status 2
adt-run1: ** bomb unexpected eof from the testbed
adt-run1: ** stop
adt-run1: ** close, scratch=/tmp/adt-run.wP0YqQ
adt-run2: >> close
adt-run1: ** bomb cannot send to testbed: ['IOError: [Errno 32] Broken pipe\n']
adt-run1: ** stop
adt-run: testbed failed: cannot send to testbed: ['IOError: [Errno 32] Broken pipe\n']
adt-run1: ** stop
adt-run2: / tests-tree rmtree /home/ubuntu/jenkins-jobs/workspace/utopic-adt-maas/results/ubtree0-tests-tree
Build step 'Execute shell' marked build as failure

Full log can be found here: http://d-jenkins.ubuntu-ci:8080/view/MAAS/job/utopic-adt-maas/984/console

This looks like bug 1317358 but the autopkgtest version used in the MAAS lab includes the fix for it, so it might be a new root cause.

[Martin Pitt (pitti)]

Indeed I'm aware of it, it occasionally happens on other autopkgtest runs, too. It's just exceptionally hard to reproduce locally. I already spent some hours trying to understand and reproduce the problem, so far not very successfully :-(

[Vincent Ladeuil (vila)]

http://d-jenkins.ubuntu-ci:8080/view/Vivid/view/BootTest/job/vivid-boottest-unity-scopes-api/1/console looks similar but I have a traceback (running with -d helps ?):

FailedConsole Output

Started by remote host 10.100.0.2
Building remotely on krillin-09 in workspace /var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api
[vivid-boottest-unity-scopes-api] $ /bin/bash -x /tmp/hudson5448376527591332470.sh
+ set +e
+ rm -Rf '*'
+ rm -r test-runner
rm: cannot remove 'test-runner': No such file or directory
+ true
+ setup_branch='lp:~canonical-ci-engineering/ubuntu-test-cases/boottest'
+ bzr_cmd='bzr branch lp:~canonical-ci-engineering/ubuntu-test-cases/boottest test-runner'
+ bzr branch 'lp:~canonical-ci-engineering/ubuntu-test-cases/boottest' test-runner
Branched 405 revisions.
+ annotate-output test-runner/scripts/boottest.sh vivid unity-scopes-api krillin-09
12:36:28 I: Started test-runner/scripts/boottest.sh vivid unity-scopes-api krillin-09
12:36:28 E: +++ readlink -f test-runner/scripts/boottest.sh
12:36:28 E: ++ dirname /var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/boottest.sh
12:36:28 E: + BASEDIR=/var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/..
12:36:28 E: + export RELEASE=vivid
12:36:28 E: + RELEASE=vivid
12:36:28 E: + export SRC_PKG_NAME=unity-scopes-api
12:36:28 E: + SRC_PKG_NAME=unity-scopes-api
12:36:28 E: + export NODE_NAME=krillin-09
12:36:28 E: + NODE_NAME=krillin-09
12:36:28 E: + export ADT_TIMEOUT=600
12:36:28 E: + ADT_TIMEOUT=600
12:36:28 E: + export RSYNC_DEST=rsync://tachash.ubuntu-ci/boottest/
12:36:28 E: + RSYNC_DEST=rsync://tachash.ubuntu-ci/boottest/
12:36:28 E: + '[' -z krillin-09 ']'
12:36:28 E: ++ /var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/../scripts/get-adb-id krillin-09
12:36:28 E: + export ANDROID_SERIAL=JW010053
12:36:28 E: + ANDROID_SERIAL=JW010053
12:36:28 E: + export VERSION=1.2.51-0ubuntu3
12:36:28 E: + VERSION=1.2.51-0ubuntu3
12:36:28 E: + BOOTTESTRC=/var/lib/jenkins/.ubuntu-ci/boottest.rc
12:36:28 E: + '[' -f /var/lib/jenkins/.ubuntu-ci/boottest.rc ']'
12:36:28 E: + ARCH=krillin
12:36:28 E: + PHABLET_PASSWORD=0000
12:36:28 E: + export SKIP_CLICK=1
12:36:28 E: + SKIP_CLICK=1
12:36:28 E: + export SKIP_TESTCONFIG=1
12:36:28 E: + SKIP_TESTCONFIG=1
12:36:28 E: + '[' -z krillin-09 ']'
12:36:28 E: + /var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/../scripts/recover.py krillin-09
12:36:28 O: /home/phablet
12:36:28 O: 0
12:36:28 E: + TESTS=/var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/../tests
12:36:28 E: + REVISION=0
12:36:28 E: + PROV_CMD='/var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/../scripts/provision.sh -r 0 -n /var/lib/jenkins/.ubuntu-ci/wifi.conf -w'
12:36:28 E: + '[' -z krillin-09 ']'
12:36:28 E: + /var/lib/jenkins/slaves/krillin-09/workspace/vivid-boottest-unity-scopes-api/test-runner/scripts/../scripts/provision.sh -r 0 -n /var/lib/jenkins/.ubuntu-ci/wifi.conf -w -s JW01005...

[Launchpad Janitor (janitor)]

This bug was fixed in the package autopkgtest - 5.0.2

---------------
autopkgtest (5.0.2) unstable; urgency=medium

  * test: Adjust ChrootRunner tests for changed apt-get source invocation.
    Commit 762397342b changed the invocation of `apt-get source` and
    `dpkg-source -x`. Adjust ChrootRunner test mock commands accordingly.

 -- Martin Pitt <email address hidden> Mon, 16 Oct 2017 11:53:55 +0200