Cannot bootstrap on azure: cannot create log collection

[Curtis Hovey]

As of commit 62e172632c3e9d8496805ed5223f9f4acc28986a juju cannot bootstrap in azure. The error could have been introduced in earlier revisions because another error was failing this and other tests. Now that the other error is fixed, we are seeing this error. This commit introduced new journalling rules so it is a possible cause.

The error is has two forms
    http://juju-ci.vapour.ws:8080/job/azure-deploy-precise-amd64/2054/console
    2014-07-31 17:32:10 INFO juju.mongo open.go:95 dialled mongo successfully on address "127.0.0.1:37017"
    2014-07-31 17:32:35 ERROR juju.cmd supercommand.go:323 failed to initialize state: cannot create log collection: read tcp 127.0.0.1:37017: i/o timeout
     2014-07-31 17:33:44 ERROR juju.provider.common bootstrap.go:120 bootstrap failed: subprocess encountered error code 1
Stopping instance...

    http://juju-ci.vapour.ws:8080/job/azure-deploy-precise-amd64/2052/console
    2014-07-31 15:47:03 INFO juju.mongo open.go:95 dialled mongo successfully on address "10.0.0.4:37017"
    2014-07-31 15:47:03 ERROR juju.cmd supercommand.go:323 failed to initialize state: cannot detect journaling: unauthorized mongo access: unauthorized
    2014-07-31 15:47:48 ERROR juju.provider.common bootstrap.go:120 bootstrap failed: subprocess encountered error code 1
Stopping instance...

[Andrew Wilkins (axwalk)]

Looks similar to issues being dealt with in the 1.20 branch.

[Martin Packman (gz)]

This is a timeout on a mongo operation creating the log collection, which is quite large (1000000 bytes), but almost certainly caused by a previous operation holding the write lock. Given bug 1351030, I'm pretty sure this is the same underlying cause and fixed by the backout.

[Curtis Hovey (sinzui)]

As of commit 9742c8d4, we can see that it happened 2 our of 3 tries on azure. Azure did pass but most of the time it failed :(

[Andrew Wilkins (axwalk)]

The bootstrap socket timeout change hasn't been forward-ported to master. That'll probably fix the i/o timeout. I don't know what causes the authorisation failure.

I'll forward port the timeout change now.

[Andrew Wilkins (axwalk)]

https://github.com/juju/juju/pull/484

[Andrew Wilkins (axwalk)]

My change has merged, which should take care of i/o timeout on bootstrap. Horacio, leaving it open and assigned to you for now; not sure if the authorisation error is still in play.

[Horacio Durán (hduran-8)]

After your patch and the reverting patch from menno I can no longer reproduce this so I will mark it as fix committed and should be reopen if someone manages to make it happen again.