intermittent failure in TestProvisionerRetriesTransientErrors

[Tim Penhey]

        panic: Session already closed

goroutine 8359 [running]:
gopkg.in/mgo%2ev2.(*Session).cluster(0xc420a604e0, 0xc420b18000)
 /workspace/src/gopkg.in/mgo.v2/session.go:1624 +0x56
gopkg.in/mgo%2ev2.copySession(0xc420a604e0, 0x413201, 0x7f890a8289c8)
 /workspace/src/gopkg.in/mgo.v2/session.go:540 +0x40
gopkg.in/mgo%2ev2.(*Session).Copy(0xc420a604e0, 0x7f890a8289c8)
 /workspace/src/gopkg.in/mgo.v2/session.go:1590 +0x40
github.com/juju/juju/state.(*database).TransactionRunner(0xc42007ec00, 0x413e18, 0xc0, 0x23d8260)
 /workspace/src/github.com/juju/juju/state/database.go:323 +0x27a
github.com/juju/juju/state.(*database).Run(0xc42007ec00, 0xc420ef00c0, 0x0, 0x0)
 /workspace/src/github.com/juju/juju/state/database.go:377 +0x41
github.com/juju/juju/state.setStatus(0x3a87e40, 0xc42007ec00, 0x2623ca1, 0x8, 0xc4208c9550, 0xc, 0x2635f19, 0x12, 0x261dfd8, 0x4, ...)
 /workspace/src/github.com/juju/juju/state/status.go:299 +0x300
github.com/juju/juju/state.(*Machine).SetInstanceStatus(0xc4203eefc0, 0x2635f19, 0x12, 0x261dfd8, 0x4, 0xc4207ffe30, 0xc42014ebe0, 0x0, 0x1)
 /workspace/src/github.com/juju/juju/state/machine.go:1055 +0x1e7
github.com/juju/juju/worker/provisioner_test.(*ProvisionerSuite).TestProvisionerRetriesTransientErrors.func1(0xc420e42ae0, 0xc4202522d0, 0xc4203eefc0, 0xc420f702d0)
 /workspace/src/github.com/juju/juju/worker/provisioner/provisioner_test.go:1540 +0x32f
created by github.com/juju/juju/worker/provisioner_test.(*ProvisionerSuite).TestProvisionerRetriesTransientErrors
 /workspace/src/github.com/juju/juju/worker/provisioner/provisioner_test.go:1526 +0x699

[Tim Penhey (thumper)]

https://github.com/juju/juju/pull/8460

[John A Meinel (jameinel)]

several intermittent failures in this test:
 bug #1753407
 bug #1754021
 bug #1749709

though they all seem slightly different. We also just saw it again in s390 in develop:
 http://10.125.0.203:8080/job/RunUnittests-s390x/277/console

[John A Meinel (jameinel)]

https://github.com/juju/juju/pull/8474

seems to be a more complete fix for avoiding the panic(). The proposed version from Tim handles when the instance succeeds, but fails if the checkStartInstance fails. (This helps prevent us from covering up the original failure.)