TestProvisionerObservesConfigChanges random failure

[Martin Packman]

A recent test run failed in worker/provisioner/provisioner_test.go but passed before and since:

<http://juju-ci.vapour.ws:8080/job/github-merge-juju/900/console>

FAIL: provisioner_test.go:890: ProvisionerSuite.TestProvisionerObservesConfigChanges
...
[LOG] 0:00.666 INFO juju.provisioner provisioner-harvest-mode is set to destroyed; unknown instances not stopped []
[LOG] 0:00.666 INFO juju.provisioner stopping known instances [0xc210578a20]
[LOG] 0:00.666 INFO juju.provisioner removing dead machine "2"
[LOG] 0:00.666 DEBUG juju.apiserver <- [2A] machine-0 {"RequestId":43,"Type":"Provisioner","Request":"Remove","Params":{"Entities":[{"Tag":"machine-2"}]}}
provisioner_test.go:913:
    // We default to the destroyed method. Only the one we know is
    // dead should be stopped; not the unknown instance.
    s.checkStopSomeInstances(c, []instance.Instance{i0}, []instance.Instance{i1})
provisioner_test.go:296:
    c.Errorf("provisioner unexpectedly stopped instance %s", instId)
... Error: provisioner unexpectedly stopped instance dummyenv-1
...

[Curtis Hovey (sinzui)]

I am raising this bug because it is failing multiple times in a day for ppc64el.

[Martin Packman (gz)]

I've seen another variety of failure on this test today:

FAIL: provisioner_test.go:1010: ProvisionerSuite.TestProvisionerObservesConfigChanges
...
[LOG] 0:05.417 INFO juju.mongo dialled mongo successfully on address "[::1]:37189"
provisioner_test.go:1036:
    c.Fatalf("PA did not action config change")
... Error: PA did not action config change

[LOG] 0:10.699 INFO juju.provisioner Shutting down provisioner task machine-0
...

[Curtis Hovey (sinzui)]

This issue is the leading cause of 386 failures and causes restests taking many hours each day.

[Aaron Bentley (abentley)]

Shown to affect 1.24:
http://reports.vapour.ws/releases/2709/job/run-unit-tests-precise-i386/attempt/1979
http://reports.vapour.ws/releases/2709/job/run-unit-tests-trusty-ppc64el/attempt/3098