MongoSuite.TestJournalDisabledDetected fails on precise amd64

[Curtis Hovey]

As of commit 62e172632c3e9d8496805ed5223f9f4acc28986a , the precise based unit tests fails after 5 retries each.
    http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-i386/442/console
    http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1336/console

The common failures are
    MongoSuite.TestJournalDisabledDetected
    MongoSuite.TestJournalEnabledDetected
    MongoSuite.TestAddRemoveSet

[Curtis Hovey (sinzui)]

This is a log from the precise i386 log.

[Eric Snow (ericsnowcurrently)]

Looks like github.com/juju/testing/mgo.go has /usr/lib/juju/bin/mongod hard-coded at one spot. My guess is that this is causing the TestJournal failures. github.com/juju/juju/mongo/mongo.go has it hard-coded as well, but the value is patched out during tests. I'll verify about mgo.go.

[Eric Snow (ericsnowcurrently)]

Introduced 7/29 by https://github.com/juju/juju/commit/62c08a9e1c444e6c4ccc220c73a6e001a6db6111.

[Eric Snow (ericsnowcurrently)]

Andrew, I run into this consistently when /usr/lib/juju does not exist. That is the case when juju has never been bootstrapped locally, right?

[Eric Snow (ericsnowcurrently)]

Incidentally, I get similar failures when I run the tests in github.com/juju/testing when /usr/lib/juju does not exist.

[Eric Snow (ericsnowcurrently)]

It's not clear to me if d047884a16ac98214a195ff1480f2c4146961d2b was meant to fix this problem. I'm guessing not.

I'm checking d379d99822f07ec9d7a4a823e2d9ed1515aa1a3d against master to see if it helps.

[Eric Snow (ericsnowcurrently)]

I meant I'm trying 2c4db889f1260c8965a88171aeba14f8b21468c2 against master.

[Eric Snow (ericsnowcurrently)]

No luck so far with 2c4db889f1260c8965a88171aeba14f8b21468c2. I'll pick this up tomorrow if it hasn't been addressed already.

[Eric Snow (ericsnowcurrently)]

Oh, and are we sure this is precise-only?

[Eric Snow (ericsnowcurrently)]

FYI, if I remove my /usr/lib/juju dir (on my trusty box), I get the exact same failures.

[Eric Snow (ericsnowcurrently)]

Here are two patches that are part of the solution. In this case we are eliminating a hard-coded path.

https://github.com/juju/testing/pull/27
https://github.com/juju/juju/pull/471

[Eric Snow (ericsnowcurrently)]

Well, over the last couple days I've been able to get my brain wrapped around this problem. After a night's sleep it has become clear to me that those two patches, while ultimately a good idea regardless (in lieu of a proper persistence abstraction), are unnecessary for this issue.

Here's a patch that attacks a different facet to get the tests passing (legitimately):

https://github.com/juju/juju/pull/476

The issue is that MongoSuite.SetUpTest() patches out $PATH, thus forcing the use of whatever mongod has been placed into /usr/lib/juju/. I'm pretty sure the intention of patching out $PATH does not apply to the two failing tests, so the simplest approach here was to simply un-patch $PATH just for the two tests.

I haven't been able to reproduce a failure for MongoSuite.TestAddRemoveSet, so I don't know yet if the patch entirely resolves this issue. However, the new patch will resolve at least a part of it.

[Martin Packman (gz)]

Menno backed out the change that introduced this test, so it's gone for now:

<https://github.com/juju/juju/pull/472>