cannot bootstrap with win-client

Bug #1293198 reported by Curtis Hovey on 2014-03-16
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
juju-core
Critical
Andrew Wilkins

Bug Description

The win-client deploy test started failing with Juju r2424. This revision just changed the version 1.17.6. We saw a similar failure when the version number changed and a tools-metadata-url was wrong. As streams.canonical.com is not updated yet, window's clients may not work until streams.canonical.com is updated. The tools do appear to be coming from aws.

Adding apt repository: deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
Running apt-get update
Running apt-get upgrade
Installing package: git
Installing package: cpu-checker
Installing package: bridge-utils
Installing package: rsyslog-gnutls
Installing package: --target-release 'precise-updates/cloud-tools' 'mongodb-server'
Fetching tools: curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o $bin/tools.tar.gz 'https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz'
Starting MongoDB server (juju-db)
Bootstrapping Juju machine agent
2014-03-16 18:44:23 ERROR juju.provider.common bootstrap.go:127 bootstrap failed: Process exited with: 1. Reason was: ()
Stopping instance...

Related branches

Curtis Hovey (sinzui) on 2014-03-17
Changed in juju-core:
milestone: none → 1.18.0
milestone: 1.18.0 → 1.17.6
Curtis Hovey (sinzui) wrote :

Streams.canonical.com is updated with 1.17.5, but the win-client bootstrap and deploy test fails.

Tim Penhey (thumper) wrote :

Which provider?

Andrew Wilkins (axwalk) wrote :

You should get a more useful error message next time, as I've landed a change to dump /var/log/cloud-init-output.log to stderr on bootstrap failure.

Curtis Hovey (sinzui) wrote :

I reconfirmed that 1.17.5 does work, installing the latest juju (1.17.6) does not. I tailed the cloud-init-out.log as the the machine was starting. The crucial issue is missing paths. The fragment of the log I captures is attached.

tools from https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz downloaded: HTTP 200; time 0.643s; size 5207140 bytes; speed 8102972.000 bytes/s 2d85ca0792407dcad2b2d2644954cadc0723590de090dd7b1881131da975803f /var/lib/juju/tools/1.17.6-precise-amd64/tools.tar.gz
...
juju-db start/running, process 6663
2014-03-18 21:33:56 DEBUG juju.environs.bootstrap state.go:73 loading "provider-state" from "https://s3.amazonaws.com/juju-gin-canasta-clue-2014-03-18-0/provider-state?AWSAccessKeyId=AKIAJFNSUVNAF5TG5UUA&Expires=1710797349&Signature=HGNrpDNG59UYKrGn9abeWeR36ZM%3D"
2014-03-18 21:33:56 ERROR juju.cmd supercommand.go:296 cannot read agent config "/var/lib/juju/agents/machine-0/agent.conf": open /var/lib/juju/agents/machine-0/agent.conf: no such file or directory

Curtis Hovey (sinzui) wrote :

I ran find repeatedly in /var/lib/juju during bootstrap. The nonce.txt exist from the start (during apt-get update) This is what find found just before shotdown:

ubuntu@ip-172-31-4-215:/var/lib/juju$ find .
.
./tools
./tools/1.17.6-precise-amd64
./tools/1.17.6-precise-amd64/downloaded-tools.txt
./tools/1.17.6-precise-amd64/jujud
./db
find: `./db': Permission denied
./server.pem
./locks
./nonce.txt
./system-identity

Curtis Hovey (sinzui) wrote :

While testing the 1.16.6, 1.17.5 and 1.17.6 I created new control-buckets to eliminate the possibility of tainted tools.

Curtis Hovey (sinzui) wrote :

I don't know what else I can gather. I could try setting termination protection on the instance as juju is setting up. The call to destroy the machine will fail, so we can do an autopsy on the machine.

Curtis Hovey (sinzui) wrote :

Attached is the debug log of the bootstrap. I don't see anything different before the error.

Installing add-apt-repository
Adding apt repository: deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
Running apt-get update
Running apt-get upgrade
Installing package: git
Installing package: cpu-checker
Installing package: bridge-utils
Installing package: rsyslog-gnutls
Installing package: --target-release 'precise-updates/cloud-tools' 'mongodb-server'
Fetching tools: curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o $bin/tools.tar.gz 'https://juju-dist.s3.amazonaws.com/testing/tools/releases/juju-1.17.6-precise-amd64.tgz'
Starting MongoDB server (juju-db)
Bootstrapping Juju machine agent
2014-03-20 02:02:23 ERROR juju.provider.common bootstrap.go:127 bootstrap failed: Process exited with: 1. Reason was: ()
Stopping instance...
2014-03-20 02:02:24 ERROR juju.provider.common bootstrap.go:141 cannot stop failed bootstrap instance "i-770eee56": The instance 'i-770eee56' may not be terminated. Modify its 'disableApiTermination' instance attribute and try again. (OperationNotPermitted)
2014-03-20 02:02:24 INFO juju.provider.common destroy.go:14 destroying environment "test-win-client"
2014-03-20 02:02:24 ERROR juju.cmd.juju common.go:19 Bootstrap failed, and the environment could not be destroyed: The instance 'i-770eee56' may not be terminated. Modify its 'disableApiTermination' instance attribute and try again. (OperationNotPermitted)
2014-03-20 02:02:24 ERROR juju.cmd supercommand.go:296 Process exited with: 1. Reason was: ()

Andrew Wilkins (axwalk) on 2014-03-20
Changed in juju-core:
assignee: nobody → Andrew Wilkins (axwalk)
status: Triaged → In Progress
Andrew Wilkins (axwalk) on 2014-03-20
Changed in juju-core:
status: In Progress → Fix Committed
Curtis Hovey (sinzui) on 2014-03-20
Changed in juju-core:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers