Agents failing, blocked by upgrade but no upgrade performed

Bug #1615986 reported by Stuart Bishop
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Invalid
Undecided
Unassigned
juju-core
Expired
Undecided
Unassigned
1.25
Expired
Undecided
Unassigned

Bug Description

With 1.25.6, on both lxd and OpenStack, I'm ending up with agents on units going into a failed state. Investigating the machine-*.log files on a failed unit, I find this at the end:

2016-08-23 08:28:07 ERROR juju.worker runner.go:223 exited "api": login for "machine-16" blocked because upgrade in progress

Note that no upgrade was performed. This was a recently deployed environment, deployed with 1.25.6. The unit was fine and last know to have reconfigured its logging, happily deploying on a very slow OpenStack but 20 mins later (08:24) failed. It appears machine 0 is deciding to upgrade itself from 1.25.6 to 1.25.6:

2016-08-23 08:04:35 DEBUG juju.network network.go:268 no lxc bridge addresses to filter for machine
2016-08-23 08:04:35 INFO juju.worker.machiner machiner.go:132 setting addresses for machine-16 to ["local-machine:127.0.0.1" "local-cloud:10.55.32.233" "local-machine:::1"]
2016-08-23 08:04:35 DEBUG juju.worker.logger logger.go:45 reconfiguring logging from "<root>=DEBUG" to "<root>=WARNING;unit=DEBUG"
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "upgrader": connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "16-container-watcher": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "apiaddressupdater": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "proxyupdater": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "deployer": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "stateconverter": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "logger": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "authenticationworker": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker.storageprovisioner storageprovisioner.go:184 connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "machiner": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "reboot": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "rsyslog": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "storageprovisioner-machine": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:212 fatal "api-post-upgrade": connection is shut down
2016-08-23 08:24:06 ERROR juju.worker runner.go:223 exited "api": connection is shut down
2016-08-23 08:24:09 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:12 ERROR juju.worker runner.go:223 exited "api": login for "machine-16" blocked because upgrade in progress
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "proxyupdater": connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "apiaddressupdater": connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "upgrader": connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.api.watcher watcher.go:84 error trying to stop watcher: connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "rsyslog": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker.storageprovisioner storageprovisioner.go:184 connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "storageprovisioner-machine": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "authenticationworker": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "machiner": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "logger": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "stateconverter": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "16-container-watcher": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "deployer": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "reboot": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:212 fatal "api-post-upgrade": connection is shut down
2016-08-23 08:27:04 ERROR juju.worker runner.go:223 exited "api": connection is shut down
2016-08-23 08:27:07 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:10 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:13 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:16 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:19 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:22 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:25 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:28 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:31 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:34 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:37 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:40 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:43 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:46 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:49 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:52 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:55 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:27:58 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:28:01 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:28:04 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:28:07 ERROR juju.worker runner.go:223 exited "api": login for "machine-16" blocked because upgrade in progress

And on machine-0:

2016-08-23 08:03:31 WARNING juju.apiserver.client status.go:679 error fetching public address: public no address
2016-08-23 08:24:07 INFO juju.cmd supercommand.go:37 running jujud [1.25.6-trusty-amd64 gc]
2016-08-23 08:24:07 DEBUG juju.agent agent.go:491 read agent config, format "1.18"
2016-08-23 08:24:07 INFO juju.cmd.jujud machine.go:419 machine agent machine-0 start (1.25.6-trusty-amd64 [gc])
2016-08-23 08:24:07 DEBUG juju.wrench wrench.go:112 couldn't read wrench directory: stat /var/lib/juju/wrench: no such f
ile or directory
2016-08-23 08:24:07 INFO juju.cmd.jujud upgrade.go:88 no upgrade steps required or upgrade steps for 1.25.6 have already
 been run.
2016-08-23 08:24:07 INFO juju.network network.go:242 setting prefer-ipv6 to false
2016-08-23 08:24:07 INFO juju.worker runner.go:269 start "api"
2016-08-23 08:24:07 DEBUG juju.worker.apicaller open.go:152 connecting with current password
2016-08-23 08:24:07 INFO juju.worker runner.go:269 start "statestarter"
2016-08-23 08:24:07 DEBUG juju.worker runner.go:196 "statestarter" started
2016-08-23 08:24:07 INFO juju.api apiclient.go:262 dialing "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf
-73e9b5e500f7/api"
2016-08-23 08:24:07 INFO juju.worker runner.go:269 start "termination"
2016-08-23 08:24:07 DEBUG juju.worker runner.go:196 "termination" started
2016-08-23 08:24:07 INFO juju.worker runner.go:269 start "state"
2016-08-23 08:24:07 INFO juju.api apiclient.go:262 dialing "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73
e9b5e500f7/api"
2016-08-23 08:24:07 INFO juju.api apiclient.go:270 error dialing "wss://10.55.32.252:17070/environment/d0720df8-1178-417
9-8adf-73e9b5e500f7/api": websocket.Dial wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api:
dial tcp 10.55.32.252:17070: connection refused
2016-08-23 08:24:07 INFO juju.api apiclient.go:270 error dialing "wss://localhost:17070/environment/d0720df8-1178-4179-8
adf-73e9b5e500f7/api": websocket.Dial wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api: dial t
cp 127.0.0.1:17070: connection refused
2016-08-23 08:24:07 DEBUG juju.worker.apicaller open.go:184 failed to connect
2016-08-23 08:24:07 INFO juju.worker runner.go:275 stopped "api", err: unable to connect to "wss://localhost:17070/envir
onment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:07 DEBUG juju.worker runner.go:203 "api" done: unable to connect to "wss://localhost:17070/environment/
d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:07 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://localhost:17070/environmen
t/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:07 INFO juju.worker runner.go:261 restarting "api" in 3s
2016-08-23 08:24:07 DEBUG juju.service discovery.go:115 discovered init system "upstart" from local host
2016-08-23 08:24:07 DEBUG juju.cmd.jujud machine.go:1409 mongodb service is installed
2016-08-23 08:24:07 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:07 INFO juju.mongo open.go:125 dialled mongo successfully on address "[::1]:37017"
2016-08-23 08:24:07 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:07 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:07 DEBUG juju.cmd.jujud machine.go:1569 replicaset member count: 1
2016-08-23 08:24:07 INFO juju.mongo mongo.go:208 Ensuring mongo server is running; data directory /var/lib/juju; port 37017
2016-08-23 08:24:07 INFO juju.mongo mongo.go:372 installing juju-mongodb
2016-08-23 08:24:07 INFO juju.utils.packaging.manager utils.go:57 Running: apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install juju-mongodb
2016-08-23 08:24:08 DEBUG juju.mongo mongo.go:331 using mongod: /usr/lib/juju/bin/mongod --version: "db version v2.4.9\nTue Aug 23 08:24:08.685 git version: nogitversion\n"
2016-08-23 08:24:08 DEBUG juju.service discovery.go:115 discovered init system "upstart" from local host
2016-08-23 08:24:08 DEBUG juju.mongo mongo.go:276 mongo exists as expected
2016-08-23 08:24:08 INFO juju.state open.go:51 opening state, mongo addresses: ["127.0.0.1:37017"]; entity machine-0
2016-08-23 08:24:08 DEBUG juju.state open.go:52 dialing mongo
2016-08-23 08:24:08 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:08 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:08 DEBUG juju.state open.go:57 connection established
2016-08-23 08:24:08 DEBUG juju.state open.go:64 mongodb login successful
2016-08-23 08:24:09 INFO juju.state state.go:195 creating lease client as machine-0
2016-08-23 08:24:09 INFO juju.state state.go:208 starting leadership manager
2016-08-23 08:24:09 INFO juju.state state.go:219 creating cloud image metadata storage
2016-08-23 08:24:09 INFO juju.state state.go:222 starting presence watcher
2016-08-23 08:24:09 DEBUG juju.state.leadership manager.go:214 waking to check leases at 2016-08-23 08:24:37.842500932 +0000 UTC
2016-08-23 08:24:09 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:09 DEBUG juju.environs imagemetadata.go:46 new user image datasource registered: environment storage
2016-08-23 08:24:09 INFO juju.worker.singular singular.go:57 runner created; isMaster true
2016-08-23 08:24:09 DEBUG juju.utils gomaxprocs.go:24 setting GOMAXPROCS to 1
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "env worker manager"
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "peergrouper"
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "env worker manager" started
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "peergrouper" started
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "restore"
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "restore" started
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "apiserver"
2016-08-23 08:24:09 DEBUG juju.cmd.jujud machine.go:1084 opening state for apistate worker
2016-08-23 08:24:09 INFO juju.state open.go:51 opening state, mongo addresses: ["127.0.0.1:37017"]; entity machine-0
2016-08-23 08:24:09 DEBUG juju.state open.go:52 dialing mongo
2016-08-23 08:24:09 INFO juju.worker.singular singular.go:100 starting "statushistorypruner"
2016-08-23 08:24:09 INFO juju.worker.singular singular.go:100 starting "txnpruner"
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "certupdater"
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "certupdater" started
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "statushistorypruner"
2016-08-23 08:24:09 INFO juju.worker runner.go:269 start "txnpruner"
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "statushistorypruner" started
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "txnpruner" started
2016-08-23 08:24:09 DEBUG juju.worker runner.go:196 "state" started
2016-08-23 08:24:09 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:09 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:09 DEBUG juju.state open.go:57 connection established
2016-08-23 08:24:09 DEBUG juju.state open.go:64 mongodb login successful
2016-08-23 08:24:10 INFO juju.worker runner.go:269 start "api"
2016-08-23 08:24:10 DEBUG juju.worker.apicaller open.go:152 connecting with current password
2016-08-23 08:24:10 INFO juju.api apiclient.go:262 dialing "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:10 INFO juju.api apiclient.go:270 error dialing "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api": websocket.Dial wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api: dial tcp 10.55.32.252:17070: connection refused
2016-08-23 08:24:10 INFO juju.api apiclient.go:262 dialing "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:10 INFO juju.api apiclient.go:270 error dialing "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api": websocket.Dial wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api: dial tcp 127.0.0.1:17070: connection refused
2016-08-23 08:24:10 DEBUG juju.worker.apicaller open.go:184 failed to connect
2016-08-23 08:24:10 INFO juju.worker runner.go:275 stopped "api", err: unable to connect to "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:10 DEBUG juju.worker runner.go:203 "api" done: unable to connect to "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:10 ERROR juju.worker runner.go:223 exited "api": unable to connect to "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:10 INFO juju.worker runner.go:261 restarting "api" in 3s
2016-08-23 08:24:10 INFO juju.state state.go:195 creating lease client as machine-0
2016-08-23 08:24:10 INFO juju.state state.go:208 starting leadership manager
2016-08-23 08:24:10 INFO juju.state state.go:219 creating cloud image metadata storage
2016-08-23 08:24:10 INFO juju.state state.go:222 starting presence watcher
2016-08-23 08:24:10 DEBUG juju.state.leadership manager.go:214 waking to check leases at 2016-08-23 08:24:37.842500932 +0000 UTC
2016-08-23 08:24:10 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:10 INFO juju.apiserver apiserver.go:168 listening on "[::]:17070"
2016-08-23 08:24:10 DEBUG juju.worker runner.go:196 "apiserver" started
2016-08-23 08:24:11 INFO juju.apiserver apiserver.go:281 [1] API connection from 10.55.32.241:54204
2016-08-23 08:24:11 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:262 <- [1] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:276 -> [1] <unknown> 10.266059ms {"RequestId":1,"Error":"login for \"machine-15\" blocked because upgrade in progress","Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:285 [1] <unknown> API connection terminated after 13.218289ms
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:281 [2] API connection from 10.55.32.248:45674
2016-08-23 08:24:12 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:262 <- [2] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:276 -> [2] <unknown> 364.95us {"RequestId":1,"Error":"login for \"machine-18\" blocked because upgrade in progress","Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:285 [2] <unknown> API connection terminated after 12.445585ms
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:281 [3] API connection from 10.55.32.237:55528
2016-08-23 08:24:12 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:262 <- [3] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:276 -> [3] <unknown> 225.574us {"RequestId":1,"Error":"login for \"machine-17\" blocked because upgrade in progress","Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:285 [3] <unknown> API connection terminated after 6.039444ms
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:281 [4] API connection from 10.55.32.233:38932
2016-08-23 08:24:12 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:262 <- [4] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:12 DEBUG juju.apiserver apiserver.go:276 -> [4] <unknown> 346.556us {"RequestId":1,"Error":"login for \"machine-16\" blocked because upgrade in progress","Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:12 INFO juju.apiserver apiserver.go:285 [4] <unknown> API connection terminated after 21.324366ms
2016-08-23 08:24:13 INFO juju.apiserver apiserver.go:281 [5] API connection from 10.172.254.15:37298
2016-08-23 08:24:13 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:13 INFO juju.worker runner.go:269 start "api"
2016-08-23 08:24:13 DEBUG juju.worker.apicaller open.go:152 connecting with current password
2016-08-23 08:24:13 INFO juju.api apiclient.go:262 dialing "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:13 INFO juju.api apiclient.go:262 dialing "wss://localhost:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:13 INFO juju.apiserver apiserver.go:281 [6] API connection from 10.55.32.252:40949
2016-08-23 08:24:13 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:13 INFO juju.api apiclient.go:194 connection established to "wss://10.55.32.252:17070/environment/d0720df8-1178-4179-8adf-73e9b5e500f7/api"
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:13 INFO juju.apiserver apiserver.go:281 [7] API connection from 127.0.0.1:58066
2016-08-23 08:24:13 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:13 INFO juju.apiserver apiserver.go:285 [7] <unknown> API connection terminated after 1.57543ms
2016-08-23 08:24:13 DEBUG juju.apiserver admin.go:149 hostPorts: [[10.55.32.252:17070 127.0.0.1:17070 [::1]:17070]]
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 71.787879ms {"RequestId":1,"Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:13 DEBUG juju.worker.apicaller open.go:191 connected
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":2,"Type":"Agent","Version":1,"Request":"GetEntities","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [5] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 3.644944ms {"RequestId":2,"Response":"'body redacted'"} Agent[""].GetEntities
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":4,"Type":"Agent","Version":1,"Request":"SetPasswords","Params":"'params redacted'"}
2016-08-23 08:24:13 INFO juju.apiserver apiserver.go:281 [8] API connection from 10.172.254.15:37296
2016-08-23 08:24:13 DEBUG juju.apiserver utils.go:71 validate env uuid: state server environment - d0720df8-1178-4179-8adf-73e9b5e500f7
2016-08-23 08:24:13 DEBUG juju.apiserver admin.go:149 hostPorts: [[10.55.32.252:17070 127.0.0.1:17070 [::1]:17070]]
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [5] user-admin@local 75.392329ms {"RequestId":1,"Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:13 INFO juju.apiserver.common password.go:74 setting mongo password for "machine-0"
2016-08-23 08:24:13 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:13 INFO juju.apiserver.common password.go:105 setting password for "machine-0"
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 310.646577ms {"RequestId":4,"Response":"'body redacted'"} Agent[""].SetPasswords
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":5,"Type":"Agent","Version":1,"Request":"GetEntities","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 2.318127ms {"RequestId":5,"Response":"'body redacted'"} Agent[""].GetEntities
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":6,"Type":"Agent","Version":1,"Request":"StateServingInfo","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 1.893209ms {"RequestId":6,"Response":"'body redacted'"} Agent[""].StateServingInfo
2016-08-23 08:24:13 INFO juju.worker runner.go:269 start "upgrader"
2016-08-23 08:24:13 DEBUG juju.worker runner.go:196 "upgrader" started
2016-08-23 08:24:13 INFO juju.worker runner.go:269 start "upgrade-steps"
2016-08-23 08:24:13 INFO juju.worker runner.go:269 start "api-post-upgrade"
2016-08-23 08:24:13 DEBUG juju.wrench wrench.go:112 couldn't read wrench directory: stat /var/lib/juju/wrench: no such file or directory
2016-08-23 08:24:13 DEBUG juju.worker runner.go:196 "upgrade-steps" started
2016-08-23 08:24:13 DEBUG juju.worker runner.go:196 "api-post-upgrade" started
2016-08-23 08:24:13 DEBUG juju.worker runner.go:196 "api" started
2016-08-23 08:24:13 INFO juju.worker runner.go:275 stopped "upgrade-steps", err: <nil>
2016-08-23 08:24:13 DEBUG juju.worker runner.go:203 "upgrade-steps" done: <nil>
2016-08-23 08:24:13 DEBUG juju.worker runner.go:206 removing "upgrade-steps" from known workers
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":7,"Type":"Upgrader","Request":"SetTools","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [8] <unknown> {"RequestId":1,"Type":"Admin","Version":2,"Request":"Login","Params":"'params redacted'"}
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:262 <- [5] user-admin@local {"RequestId":3,"Type":"Client","Request":"FullStatus","Params":"'params redacted'"}
2016-08-23 08:24:13 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:13 DEBUG juju.apiserver.client status.go:123 Services: map[client:client postgresql:postgresql]
2016-08-23 08:24:13 DEBUG juju.apiserver admin.go:149 hostPorts: [[10.55.32.252:17070 127.0.0.1:17070 [::1]:17070]]
2016-08-23 08:24:13 DEBUG juju.apiserver apiserver.go:276 -> [8] user-admin@local 93.489082ms {"RequestId":1,"Response":"'body redacted'"} Admin[""].Login
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [5] user-admin@local 92.513183ms {"RequestId":3,"Response":"'body redacted'"} Client[""].FullStatus
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 271.822371ms {"RequestId":7,"Response":"'body redacted'"} Upgrader[""].SetTools
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":8,"Type":"Upgrader","Request":"WatchAPIVersion","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 2.853607ms {"RequestId":8,"Response":"'body redacted'"} Upgrader[""].WatchAPIVersion
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":9,"Type":"Upgrader","Request":"DesiredVersion","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":10,"Type":"NotifyWatcher","Id":"2","Request":"Next","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 3.490731ms {"RequestId":9,"Response":"'body redacted'"} Upgrader[""].DesiredVersion
2016-08-23 08:24:14 INFO juju.worker.upgrader upgrader.go:154 desired tool version: 1.25.6
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "env worker manager"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "peergrouper"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "restore"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "certupdater"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "statushistorypruner"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "txnpruner"
2016-08-23 08:24:14 DEBUG juju.cmd.jujud machine.go:1667 upgrades done, starting worker "api-post-upgrade"
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "proxyupdater"
2016-08-23 08:24:14 DEBUG juju.worker.proxyupdater proxyupdater.go:79 write system files: true
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "proxyupdater" started
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "resumer"
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "resumer" started
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 DEBUG juju.worker.certupdater certupdater.go:118 new machine addresses: []network.Address{local-cloud:10.55.32.252(canonistack)}
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":11,"Type":"Environment","Request":"EnvironConfig","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":12,"Type":"ProxyUpdater","Version":1,"Request":"ProxyConfig","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 63.988666ms {"RequestId":11,"Response":"'body redacted'"} Environment[""].EnvironConfig
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "d0720df8-1178-4179-8adf-73e9b5e500f7"
2016-08-23 08:24:14 INFO juju.state open.go:51 opening state, mongo addresses: ["127.0.0.1:37017"]; entity machine-0
2016-08-23 08:24:14 DEBUG juju.state open.go:52 dialing mongo
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 51.67865ms {"RequestId":12,"Response":"'body redacted'"} ProxyUpdater[""].ProxyConfig
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "machiner"
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "reboot"
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "machiner" started
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "reboot" started
2016-08-23 08:24:14 DEBUG juju.worker.reboot reboot.go:54 Reboot worker setup
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "apiaddressupdater"
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "apiaddressupdater" started
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "logger"
2016-08-23 08:24:14 DEBUG juju.worker.logger logger.go:35 initial log config: "<root>=DEBUG"
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "rsyslog"
2016-08-23 08:24:14 DEBUG juju.worker.rsyslog worker.go:108 starting rsyslog worker mode 2 for "machine-0" ""
2016-08-23 08:24:14 DEBUG juju.worker.logger logger.go:60 logger setup
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "logger" started
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "rsyslog" started
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "diskmanager"
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "diskmanager" started
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "storageprovisioner-machine"
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "networker"
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "storageprovisioner-machine" started
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "networker" started
2016-08-23 08:24:14 INFO juju.networker networker.go:163 networker is disabled - not starting on machine "machine-0"
2016-08-23 08:24:14 INFO juju.worker runner.go:269 start "authenticationworker"
2016-08-23 08:24:14 DEBUG juju.worker.peergrouper worker.go:432 found new machine "0"
2016-08-23 08:24:14 DEBUG juju.worker.proxyupdater proxyupdater.go:161 new proxy settings proxy.Settings{Http:"", Https:"", Ftp:"", NoProxy:"10.55.32.252,127.0.0.1,::1"}
2016-08-23 08:24:14 INFO juju.worker runner.go:275 stopped "networker", err: <nil>
2016-08-23 08:24:14 DEBUG juju.worker runner.go:196 "authenticationworker" started
2016-08-23 08:24:14 DEBUG juju.worker runner.go:203 "networker" done: <nil>
2016-08-23 08:24:14 DEBUG juju.worker runner.go:206 removing "networker" from known workers
2016-08-23 08:24:14 INFO juju.apiserver apiserver.go:285 [5] user-admin@local API connection terminated after 1.049061379s
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [8] user-admin@local {"RequestId":3,"Type":"Client","Request":"FullStatus","Params":"'params redacted'"}
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "127.0.0.1:37017"
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":13,"Type":"Machiner","Request":"Life","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":14,"Type":"Reboot","Version":1,"Request":"WatchForRebootEvent","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":15,"Type":"Machiner","Request":"WatchAPIHostPorts","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":16,"Type":"Logger","Request":"LoggingConfig","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":17,"Type":"Rsyslog","Request":"GetRsyslogConfig","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":18,"Type":"StorageProvisioner","Version":1,"Request":"WatchForEnvironConfigChanges","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":19,"Type":"KeyUpdater","Request":"AuthorisedKeys","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 100.661653ms {"RequestId":13,"Response":"'body redacted'"} Machiner[""].Life
2016-08-23 08:24:14 DEBUG juju.worker.proxyupdater proxyupdater.go:180 new apt proxy settings proxy.Settings{Http:"", Https:"", Ftp:"", NoProxy:""}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 39.854443ms {"RequestId":14,"Response":"'body redacted'"} Reboot[""].WatchForRebootEvent
2016-08-23 08:24:14 DEBUG juju.worker.certupdater certupdater.go:190 existing cert addresses map[]
2016-08-23 08:24:14 DEBUG juju.worker.certupdater certupdater.go:191 new addresses [localhost juju-apiserver juju-mongodb anything 10.55.32.252]
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":20,"Type":"ProxyUpdater","Version":1,"Request":"WatchForProxyConfigAndAPIHostPortChanges","Params":"'params redacted'"}
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 DEBUG juju.network network.go:268 no lxc bridge addresses to filter for machine
2016-08-23 08:24:14 INFO juju.worker.machiner machiner.go:132 setting addresses for machine-0 to ["local-machine:127.0.0.1" "local-cloud:10.55.32.252" "local-machine:::1"]
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 63.895072ms {"RequestId":16,"Response":"'body redacted'"} Logger[""].LoggingConfig
2016-08-23 08:24:14 DEBUG juju.state open.go:57 connection established
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:276 -> [6] machine-0 66.734068ms {"RequestId":15,"Response":"'body redacted'"} Machiner[""].WatchAPIHostPorts
2016-08-23 08:24:14 INFO juju.worker.diskmanager diskmanager.go:62 block devices changed: [{vda [] 10240 true } {vdb [/dev/disk/by-label/ephemeral0 /dev/disk/by-uuid/9f6f8a8f-3721-45ee-a011-01c6fc404766] ephemeral0 9f6f8a8f-3721-45ee-a011-01c6fc404766 20480 ext3 true /mnt}]
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 INFO juju.mongo open.go:125 dialled mongo successfully on address "10.55.32.252:37017"
2016-08-23 08:24:14 DEBUG juju.container.kvm kvm.go:71 kvm-ok output:
INFO: /dev/kvm does not exist
HINT: sudo modprobe kvm_amd
INFO: Your CPU supports KVM extensions
KVM acceleration can be used

2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":21,"Type":"Machiner","Request":"SetMachineAddresses","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":22,"Type":"Reboot","Version":1,"Request":"GetRebootAction","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":23,"Type":"NotifyWatcher","Id":"3","Request":"Next","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":24,"Type":"DiskManager","Version":1,"Request":"SetMachineBlockDevices","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.apiserver apiserver.go:262 <- [6] machine-0 {"RequestId":25,"Type":"Provisioner","Version":1,"Request":"Life","Params":"'params redacted'"}
2016-08-23 08:24:14 DEBUG juju.state open.go:64 mongodb login successful
2016-08-23 08:24:14 DEBUG juju.worker.logger logger.go:45 reconfiguring logging from "<root>=DEBUG" to "<root>=WARNING;unit=DEBUG"
2016-08-23 08:28:02 INFO juju.cmd supercommand.go:37 running jujud [1.25.6-trusty-amd64 gc]
2016-08-23 08:28:02 DEBUG juju.agent agent.go:491 read agent config, format "1.18"
2016-08-23 08:28:02 INFO juju.cmd.jujud machine.go:419 machine agent machine-0 start (1.25.6-trusty-amd64 [gc])
2016-08-23 08:28:02 DEBUG juju.wrench wrench.go:112 couldn't read wrench directory: stat /var/lib/juju/wrench: no such file or directory
2016-08-23 08:28:02 INFO juju.cmd.jujud upgrade.go:88 no upgrade steps required or upgrade steps for 1.25.6 have already been run.

Tags: canonical-is
Stuart Bishop (stub)
tags: added: canonical-is
Revision history for this message
Stuart Bishop (stub) wrote :
Revision history for this message
Stuart Bishop (stub) wrote :
Revision history for this message
Stuart Bishop (stub) wrote :
Revision history for this message
Stuart Bishop (stub) wrote :
Curtis Hovey (sinzui)
affects: juju → juju-core
Revision history for this message
Stuart Bishop (stub) wrote :

I haven't been able to reproduce this after downgrading and forcing 1.25.5

no longer affects: juju-core
Revision history for this message
Anastasia (anastasia-macmood) wrote :

Stuart,

You downgraded and forced 1.25.5. Did you then upgrade to 1.25.6? Or are you on 1.25.5?

Changed in juju-core:
status: New → Incomplete
Changed in juju:
status: New → Invalid
Revision history for this message
Stuart Bishop (stub) wrote : Re: [Bug 1615986] Re: Agents failing, blocked by upgrade but no upgrade performed

On 25 August 2016 at 09:01, Anastasia <email address hidden>
wrote:

> Stuart,
>
> You downgraded and forced 1.25.5. Did you then upgrade to 1.25.6? Or are
> you on 1.25.5?
>
>
I'm on 1.25.5. I have only triggered the problem with 1.25.6, or when I
forgot to bootstrap 1.25.5 with --no-auto-upgrade

--
Stuart Bishop <email address hidden>

Changed in juju-core:
status: Incomplete → Triaged
importance: Undecided → Critical
status: Triaged → Invalid
importance: Critical → Undecided
Changed in juju-core:
status: Invalid → Triaged
importance: Undecided → Critical
Changed in juju-core:
milestone: none → 1.25.11
Revision history for this message
Anastasia (anastasia-macmood) wrote :

@Stuart,

Are still affected by this bug?
Have you upgraded to/bootstrapped later 1.25.x? 1.25.10 is currently out.

Some similar bugs were fixed in the releases since, eg. bug # 1544796

Changed in juju-core:
status: Triaged → Incomplete
importance: Critical → Undecided
milestone: 1.25.11 → none
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for juju-core because there has been no activity for 60 days.]

Changed in juju-core:
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for juju-core 1.25 because there has been no activity for 60 days.]

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.