cmd/juju/commands: not all tests are being run

[Andrew Wilkins]

I just ran the tests in the "cmd/juju/commands" package, and expected one to fail. It didn't. Tail of the output for that package:

    PASS: machine_test.go:35: MachineSuite.TestMachineAdd 0.014s
    PASS: machine_test.go:48: MachineSuite.TestMachineRemove 0.017s
    PASS: main_test.go:154: MainSuite.TestActualRunJujuArgOrder 0.323s
    PASS: main_test.go:621: MainSuite.TestAllCommandsPurposeDocCapitalization 0.000s
    ok github.com/juju/juju/cmd/juju/commands 52.096s

Spot the difference with the output from a different package:

    PASS: metricsdebug_test.go:134: DebugMetricsCommandSuite.TestServiceWithNoptionGreaterThanMetricCount 0.067s
    PASS: metricsdebug_test.go:166: DebugMetricsCommandSuite.TestUnitJsonOutput 0.050s
    PASS: metricsdebug_test.go:90: DebugMetricsCommandSuite.TestUnits 0.070s
    OK: 10 passed
    PASS
    ok github.com/juju/juju/cmd/juju/metricsdebug 1.036s

Something is most likely calling os.Exit(0).

[Andrew Wilkins (axwalk)]

Here's the culprit:

START: main_test.go:179: MainSuite.TestFirstRun2xFrom1x
START: <autogenerated>:872: MainSuite.SetUpTest
PASS: <autogenerated>:872: MainSuite.SetUpTest 0.000s

ok github.com/juju/juju/cmd/juju/commands 0.756s

[Andrew Wilkins (axwalk)]

Fixed by https://github.com/juju/juju/pull/5345

Nate also has a PR up for review (https://github.com/juju/juju/pull/5250) which addresses this, but also does other things that we may not want to force through CI right now.