cannot bootstrap with win-client

[Curtis Hovey]

The win-client deploy test started failing with Juju r2424. This revision just changed the version 1.17.6. We saw a similar failure when the version number changed and a tools-metadata-url was wrong. As streams.canonical.com is not updated yet, window's clients may not work until streams.canonical.com is updated. The tools do appear to be coming from aws.

Adding apt repository: deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
Running apt-get update
Running apt-get upgrade
Installing package: git
Installing package: cpu-checker
Installing package: bridge-utils
Installing package: rsyslog-gnutls
Installing package: --target-release 'precise-updates/cloud-tools' 'mongodb-server'
Fetching tools: curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o $bin/tools.tar.gz 'https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz'
Starting MongoDB server (juju-db)
Bootstrapping Juju machine agent
2014-03-16 18:44:23 ERROR juju.provider.common bootstrap.go:127 bootstrap failed: Process exited with: 1. Reason was: ()
Stopping instance...

[Curtis Hovey (sinzui)]

Streams.canonical.com is updated with 1.17.5, but the win-client bootstrap and deploy test fails.

[Tim Penhey (thumper)]

Which provider?

[Andrew Wilkins (axwalk)]

You should get a more useful error message next time, as I've landed a change to dump /var/log/cloud-init-output.log to stderr on bootstrap failure.

[Curtis Hovey (sinzui)]

I reconfirmed that 1.17.5 does work, installing the latest juju (1.17.6) does not. I tailed the cloud-init-out.log as the the machine was starting. The crucial issue is missing paths. The fragment of the log I captures is attached.

tools from https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz downloaded: HTTP 200; time 0.643s; size 5207140 bytes; speed 8102972.000 bytes/s 2d85ca0792407dcad2b2d2644954cadc0723590de090dd7b1881131da975803f /var/lib/juju/tools/1.17.6-precise-amd64/tools.tar.gz
...
juju-db start/running, process 6663
2014-03-18 21:33:56 DEBUG juju.environs.bootstrap state.go:73 loading "provider-state" from "https://s3.amazonaws.com/juju-gin-canasta-clue-2014-03-18-0/provider-state?AWSAccessKeyId=AKIAJFNSUVNAF5TG5UUA&Expires=1710797349&Signature=HGNrpDNG59UYKrGn9abeWeR36ZM%3D"
2014-03-18 21:33:56 ERROR juju.cmd supercommand.go:296 cannot read agent config "/var/lib/juju/agents/machine-0/agent.conf": open /var/lib/juju/agents/machine-0/agent.conf: no such file or directory

[Curtis Hovey (sinzui)]

I ran find repeatedly in /var/lib/juju during bootstrap. The nonce.txt exist from the start (during apt-get update) This is what find found just before shotdown:

ubuntu@ip-172-31-4-215:/var/lib/juju$ find .
.
./tools
./tools/1.17.6-precise-amd64
./tools/1.17.6-precise-amd64/downloaded-tools.txt
./tools/1.17.6-precise-amd64/jujud
./db
find: `./db': Permission denied
./server.pem
./locks
./nonce.txt
./system-identity

[Curtis Hovey (sinzui)]

While testing the 1.16.6, 1.17.5 and 1.17.6 I created new control-buckets to eliminate the possibility of tainted tools.

[Curtis Hovey (sinzui)]

I don't know what else I can gather. I could try setting termination protection on the instance as juju is setting up. The call to destroy the machine will fail, so we can do an autopsy on the machine.

[Curtis Hovey (sinzui)]

Attached is the debug log of the bootstrap. I don't see anything different before the error.

Installing add-apt-repository
Adding apt repository: deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
Running apt-get update
Running apt-get upgrade
Installing package: git
Installing package: cpu-checker
Installing package: bridge-utils
Installing package: rsyslog-gnutls
Installing package: --target-release 'precise-updates/cloud-tools' 'mongodb-server'
Fetching tools: curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o $bin/tools.tar.gz 'https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz'
Starting MongoDB server (juju-db)
Bootstrapping Juju machine agent
2014-03-20 02:02:23 ERROR juju.provider.common bootstrap.go:127 bootstrap failed: Process exited with: 1. Reason was: ()
Stopping instance...
2014-03-20 02:02:24 ERROR juju.provider.common bootstrap.go:141 cannot stop failed bootstrap instance "i-770eee56": The instance 'i-770eee56' may not be terminated. Modify its 'disableApiTermination' instance attribute and try again. (OperationNotPermitted)
2014-03-20 02:02:24 INFO juju.provider.common destroy.go:14 destroying environment "test-win-client"
2014-03-20 02:02:24 ERROR juju.cmd.juju common.go:19 Bootstrap failed, and the environment could not be destroyed: The instance 'i-770eee56' may not be terminated. Modify its 'disableApiTermination' instance attribute and try again. (OperationNotPermitted)
2014-03-20 02:02:24 ERROR juju.cmd supercommand.go:296 Process exited with: 1. Reason was: ()