Vivid unit tests need to pass

[Curtis Hovey]

We have seen the vivid unit tests pass, but that id only 1/10. We need to get this suite reliable for 1.23.0. The common test failures are:
FAIL: pinger_test.go:97: pingerSuite.TestAgentConnectionDelaysShutdownWithPing
FAIL: root_test.go:40: rootSuite.TestPingTimeout
FAIL: replicaset_test.go:157: MongoSuite.TestAddRemoveSet (bug 1398415)
FAIL: tracker_test.go:406: TrackerSuite.TestWaitMinionNeverBecomeMinion (bug 1433384)

The pingerSuite.TestAgentConnectionDelaysShutdownWithPing and rootSuite.TestPingTimeout failures are odd. We commonly see these tests fail when the tests are run in lxc, but the vivid host is a 8G RAM 30G disk machine with 2 CPUs. Vivid uses go1.3.3

[Curtis Hovey (sinzui)]

Attached is a log of the run.

[Eric Snow (ericsnowcurrently)]

This appears to be related to https://bugs.launchpad.net/juju-core/+bug/1433384.

[Martin Packman (gz)]

Another test run:

<http://data.vapour.ws/juju-ci/products/version-2478/run-unit-tests-vivid-amd64/build-289/consoleText>

logs_test.go:109: LogsSuite.TestPruneLogsBySize
worker_test.go:89: suite.TestPrunesLogsBySize
worker_test.go:54: suite.TestPrunesOldLogs

(all bug 1434741 and fixed on trunk)

resumer_test.go:37: ResumerSuite.TestResumerCalls
resumer_test.go:61:
    c.Assert(diff <= 4*testInterval, jc.IsTrue)
... obtained bool = false

No current bug filed? Also intermittent?

[Eric Snow (ericsnowcurrently)]

I've addressed lp:1408459 but there are other failures that need to be addressed. I expect we will bump into various transient failures as time goes by, not unlike what happens with merge-gating and with the other unit tests. Regardless, I'm aiming to at least get the test runs consistently passing ASAP, and certainly for the 1.23.0 release.

[Eric Snow (ericsnowcurrently)]

I've opened lp:1436397 for one of the failures. Note that it does not affect 1.23 (only master).

[Eric Snow (ericsnowcurrently)]

I've opened lp:1436407 for the (currently) only remaining test failure on 1.23.

[Eric Snow (ericsnowcurrently)]

intermittent on 1.23: lp:1398415 (old bug)

[Eric Snow (ericsnowcurrently)]

a possibly intermittent failure on 1.23: lp:1436495 (new bug)

[Eric Snow (ericsnowcurrently)]

I'd meant to say *1.24* on that last one.

[Eric Snow (ericsnowcurrently)]

Another possibly intermittent failure on 1.24: lp:1436507 (new bug)

[Eric Snow (ericsnowcurrently)]

Both 1.23 and master are now passing intermittently. Presumably we still need to get them passing much more consistenly.

[Eric Snow (ericsnowcurrently)]

On master the tests should consistently pass once the proposed fix for lp:1436397 lands (probably today).

[Eric Snow (ericsnowcurrently)]

I've logged another intermittent one for master: lp:1438274.

[Eric Snow (ericsnowcurrently)]

The tests are passing frequently enough that I'm closing this issue. An further specific failures can be addressed in new issues.