tests which kill ssh (ssh exit code 255) loop forever

[Michael Hudson-Doyle]

Currently http://people.canonical.com/~ubuntu-archive/proposed-migration/yakkety/update_excuses.html for docker.io (1.10.3-0ubuntu6 to 1.10.3-0ubuntu7) says "ppc64el always failed" and the "always failed" links to https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-yakkety/yakkety/ppc64el/d/docker.io/20160428_134742@/log.gz but there is no entry for on 1.10.3-0ubuntu7 http://autopkgtest.ubuntu.com/packages/d/docker.io/yakkety/ppc64el/

I don't know if it's related, but the docker.io tests seem to have been running over and over over the last several hours. E.g. when I got up this morning, docker.io was on running.shtml as running on amd64 for 35 mins, even though I had seen docker.io as running on amd64 on the same page when I went to bed 8 or so hours before.

[Martin Pitt (pitti)]

That's right, the docker.io test is currently looping at least on amd64 (possibly ppc64el too), as it kills the server-side end of the ssh connection. We currently treat this as a temporary failure and restart the test then. I generally want to keep this behaviour to guard against cloud or test flakiness, but we need to dial that back a bit to not do an endless loop. Independently of that, docker.io (or whatever causes it) should be fixed to not kill sshd.

[Martin Pitt (pitti)]

The docker.io test failure was reported to bug 1576419. I thought about the loop above, and I don't really see a way to prevent the auto-retry that also wouldn't cause infra-related test failures for other tests, unless we do a lot of state keeping (which is impractical).

So for now I blacklisted yakkety/docker.io/amd64 in https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/commit/?id=846938e789c1, which will stop the eternal loop.