2024-04-16T08:06:56.937Z [pebble] HTTP API server listening on ":38811". 2024-04-16T08:06:56.937Z [pebble] Started daemon. 2024-04-16T08:06:56.944Z [pebble] POST /v1/services 7.096804ms 202 2024-04-16T08:06:56.945Z [pebble] Started default services with change 3. 2024-04-16T08:06:56.947Z [pebble] Service "jujud" starting: /var/lib/juju/tools/jujud machine --data-dir /var/lib/juju --controller-id 0 --log-to-stderr --show-log 2024-04-16T08:06:56.973Z [jujud] 2024-04-16 08:06:56 INFO juju.cmd supercommand.go:56 running jujud [3.4.2 a80becbb4da5985fa53c63824a4bd809e9d03954 gc go1.20.14] 2024-04-16T08:06:56.978Z [jujud] 2024-04-16 08:06:56 INFO juju.worker.upgradesteps worker.go:60 upgrade steps for 3.4.2 have already been run. 2024-04-16T08:06:56.978Z [jujud] 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-exec" as exsting path has a normal file 2024-04-16T08:06:56.978Z [jujud] 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-dumplogs" as exsting path has a normal file 2024-04-16T08:06:56.979Z [jujud] 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-introspect" as exsting path has a normal file 2024-04-16T08:06:56.979Z [jujud] 2024-04-16 08:06:56 INFO juju.cmd.jujud runner.go:592 start "engine" 2024-04-16T08:06:57.020Z [jujud] 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:379 host is configured to use loopback address as a Dqlite node 2024-04-16T08:06:57.815Z [jujud] 2024-04-16 08:06:57 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [a0a72c] "controller-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused 2024-04-16T08:06:57.815Z [jujud] 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:477 serving Dqlite application (ID: 3297041220608546238) 2024-04-16T08:06:57.816Z [jujud] 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:481 current cluster: []protocol.NodeInfo{protocol.NodeInfo{ID:0x2dc171858c3155be, Address:"127.0.0.1:17666", Role:0}} 2024-04-16T08:06:58.038Z [jujud] 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:06:58.046Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:06:58.046Z [jujud] 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:06:58.046Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-a0a72c11-233c-4cb9-898f-69d9b2f0537c successfully 2024-04-16T08:06:58.046Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:592 start "txnlog" 2024-04-16T08:06:58.047Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher txnwatcher.go:317 txn watcher started 2024-04-16T08:06:58.048Z [jujud] 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:06:58.054Z [jujud] 2024-04-16 08:06:58 INFO juju.cmd.jujud machine.go:977 juju database opened 2024-04-16T08:06:58.054Z [jujud] 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:06:58.054Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:06:58.054Z [jujud] 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:06:58.054Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-946658c9-f947-48cb-89e1-cfc55ab360ac successfully 2024-04-16T08:06:58.064Z [jujud] 2024-04-16 08:06:58 INFO juju.state logs.go:177 ensuring logs collection for a0a72c11-233c-4cb9-898f-69d9b2f0537c, capped at 20 MiB 2024-04-16T08:06:58.072Z [jujud] 2024-04-16 08:06:58 ERROR core.auditlog auditlog.go:196 Unable to prime /var/log/juju/audit.log (proceeding anyway): chown /var/log/juju/audit.log: operation not permitted 2024-04-16T08:06:58.109Z [jujud] 2024-04-16 08:06:58 INFO juju.state logs.go:177 ensuring logs collection for 946658c9-f947-48cb-89e1-cfc55ab360ac, capped at 20 MiB 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-a0a72c11-233c-4cb9-898f-69d9b2f0537c successfully 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:592 start "txnlog" 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:410 runner is dying 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:623 stopped "txnlog", err: 2024-04-16T08:06:58.123Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:410 runner is dying 2024-04-16T08:06:58.124Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher txnwatcher.go:317 txn watcher started 2024-04-16T08:06:58.124Z [jujud] 2024-04-16 08:06:58 WARNING juju.state.pool.txnwatcher txnwatcher.go:338 txn watcher sync error: tomb: dying 2024-04-16T08:06:58.124Z [jujud] 2024-04-16 08:06:58 WARNING juju.state.pool.txnwatcher txnwatcher.go:346 txn watcher resume queued 2024-04-16T08:06:58.125Z [jujud] 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:623 stopped "txnlog", err: 2024-04-16T08:06:58.145Z [jujud] 2024-04-16 08:06:58 INFO juju.state.allwatcher allwatcher.go:1819 allwatcher loaded for model "a0a72c11-233c-4cb9-898f-69d9b2f0537c" in 81.734063ms 2024-04-16T08:06:58.151Z [jujud] 2024-04-16 08:06:58 INFO juju.state.allwatcher allwatcher.go:1819 allwatcher loaded for model "946658c9-f947-48cb-89e1-cfc55ab360ac" in 5.885948ms 2024-04-16T08:06:58.199Z [jujud] 2024-04-16 08:06:58 INFO juju.worker.httpserver worker.go:315 listening on "[::]:17070" 2024-04-16T08:06:58.818Z [jujud] 2024-04-16 08:06:58 INFO juju.worker.leaseexpiry worker.go:123 expired 5 leases 2024-04-16T08:07:00.951Z [jujud] 2024-04-16 08:07:00 INFO juju.apiserver.connection request_notifier.go:96 agent login: unit-controller-0 for a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:06:56 INFO juju.cmd supercommand.go:56 running containerAgent [3.4.2 a80becbb4da5985fa53c63824a4bd809e9d03954 gc go1.21.9] 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:06:56 INFO juju.cmd.containeragent.unit runner.go:592 start "unit" 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:06:56 INFO juju.worker.upgradesteps worker.go:60 upgrade steps for 3.4.2 have already been run. 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:06:56 INFO juju.worker.probehttpserver server.go:157 starting http server on 127.0.0.1:65301 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:06:57 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [a0a72c] "unit-controller-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/a0a72c11-233c-4cb9-898f-69d9b2f0537c/api" 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 INFO juju.worker.apicaller connect.go:163 [a0a72c] "unit-controller-0" successfully connected to "localhost:17070" 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 INFO juju.worker.migrationminion worker.go:142 migration phase is now: NONE migration 2024-04-16T08:07:00.988Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 INFO juju.worker.logger logger.go:120 logger worker started 2024-04-16T08:07:00.991Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 WARNING juju.worker.proxyupdater proxyupdater.go:241 unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" 2024-04-16T08:07:00.991Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:00 ERROR juju.worker.proxyupdater proxyupdater.go:279 error writing apt proxy config file: open /etc/apt/apt.conf.d/95-juju-proxy-settings: permission denied 2024-04-16T08:07:01.005Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.agent.tools symlinks.go:20 ensure jujuc symlinks in /var/lib/juju/tools/unit-controller-0 2024-04-16T08:07:01.009Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.leadership tracker.go:194 controller/0 promoted to leadership of controller 2024-04-16T08:07:01.014Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.uniter uniter.go:363 unit "controller/0" started 2024-04-16T08:07:01.017Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.uniter uniter.go:389 hooks are retried true 2024-04-16T08:07:01.017Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:01.017Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:01.085Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.uniter resolver.go:76 reboot detected; triggering implicit start hook to notify charm 2024-04-16T08:07:01.269Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO unit.controller/0.juju-log server.go:325 Running legacy hooks/start. 2024-04-16T08:07:01.590Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:01 INFO juju.worker.uniter.operation runhook.go:186 ran "start" hook (via hook dispatching script: dispatch) 2024-04-16T08:07:02.552Z [jujud] 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/a0a72c11-233c-4cb9-898f-69d9b2f0537c/api" 2024-04-16T08:07:02.553Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.apicaller connect.go:163 [a0a72c] "controller-0" successfully connected to "localhost:17070" 2024-04-16T08:07:02.575Z [jujud] 2024-04-16 08:07:02 INFO juju.apiserver.common password.go:75 setting mongo password for "controller-0" 2024-04-16T08:07:02.598Z [jujud] 2024-04-16 08:07:02 INFO juju.apiserver.common password.go:100 setting password for "controller-0" 2024-04-16T08:07:02.625Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:02.625Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:02.652Z [jujud] 2024-04-16 08:07:02 INFO juju.workers.modelworkermanager runner.go:592 start "a0a72c11-233c-4cb9-898f-69d9b2f0537c" 2024-04-16T08:07:02.653Z [jujud] 2024-04-16 08:07:02 INFO juju.workers.modelworkermanager runner.go:592 start "946658c9-f947-48cb-89e1-cfc55ab360ac" 2024-04-16T08:07:02.670Z [jujud] 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/946658c9-f947-48cb-89e1-cfc55ab360ac/api" 2024-04-16T08:07:02.671Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.migrationminion worker.go:142 migration phase is now: NONE 2024-04-16T08:07:02.671Z [jujud] 2024-04-16 08:07:02 INFO juju.agent identity.go:28 removing system identity file 2024-04-16T08:07:02.672Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.logger logger.go:120 logger worker started 2024-04-16T08:07:02.674Z [jujud] 2024-04-16 08:07:02 WARNING juju.worker.proxyupdater proxyupdater.go:241 unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" 2024-04-16T08:07:02.675Z [jujud] 2024-04-16 08:07:02 ERROR juju.worker.proxyupdater proxyupdater.go:279 error writing apt proxy config file: open /etc/apt/apt.conf.d/95-juju-proxy-settings: read-only file system 2024-04-16T08:07:02.678Z [jujud] 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/a0a72c11-233c-4cb9-898f-69d9b2f0537c/api" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.cmd supercommand.go:56 running jujud [3.4.2 a80becbb4da5985fa53c63824a4bd809e9d03954 gc go1.20.14] 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.worker.upgradesteps worker.go:60 upgrade steps for 3.4.2 have already been run. 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-exec" as exsting path has a normal file 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-dumplogs" as exsting path has a normal file 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.cmd.jujud machine.go:1272 skipping creating symlink "/usr/bin/juju-introspect" as exsting path has a normal file 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:56 INFO juju.cmd.jujud runner.go:592 start "engine" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:379 host is configured to use loopback address as a Dqlite node 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:57 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [a0a72c] "controller-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:477 serving Dqlite application (ID: 3297041220608546238) 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:57 INFO juju.worker.dbaccessor worker.go:481 current cluster: []protocol.NodeInfo{protocol.NodeInfo{ID:0x2dc171858c3155be, Address:"127.0.0.1:17666", Role:0}} 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-a0a72c11-233c-4cb9-898f-69d9b2f0537c successfully 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:592 start "txnlog" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher txnwatcher.go:317 txn watcher started 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:07:02.679Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.apicaller connect.go:163 [a0a72c] "controller-0" successfully connected to "localhost:17070" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.cmd.jujud machine.go:977 juju database opened 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state open.go:192 creating cloud image metadata storage 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-946658c9-f947-48cb-89e1-cfc55ab360ac successfully 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state logs.go:177 ensuring logs collection for a0a72c11-233c-4cb9-898f-69d9b2f0537c, capped at 20 MiB 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 ERROR core.auditlog auditlog.go:196 Unable to prime /var/log/juju/audit.log (proceeding anyway): chown /var/log/juju/audit.log: operation not permitted 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state logs.go:177 ensuring logs collection for 946658c9-f947-48cb-89e1-cfc55ab360ac, capped at 20 MiB 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:453 starting standard state workers 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:592 start "txnlog" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state state.go:459 started state workers for model-a0a72c11-233c-4cb9-898f-69d9b2f0537c successfully 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:592 start "txnlog" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:410 runner is dying 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.watcher runner.go:623 stopped "txnlog", err: 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:410 runner is dying 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher txnwatcher.go:317 txn watcher started 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 WARNING juju.state.pool.txnwatcher txnwatcher.go:338 txn watcher sync error: tomb: dying 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 WARNING juju.state.pool.txnwatcher txnwatcher.go:346 txn watcher resume queued 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.pool.txnwatcher runner.go:623 stopped "txnlog", err: 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.allwatcher allwatcher.go:1819 allwatcher loaded for model "a0a72c11-233c-4cb9-898f-69d9b2f0537c" in 81.734063ms 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.state.allwatcher allwatcher.go:1819 allwatcher loaded for model "946658c9-f947-48cb-89e1-cfc55ab360ac" in 5.885948ms 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.worker.httpserver worker.go:315 listening on "[::]:17070" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:06:58 INFO juju.worker.leaseexpiry worker.go:123 expired 5 leases 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:00 INFO juju.apiserver.connection request_notifier.go:96 agent login: unit-controller-0 for a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/a0a72c11-233c-4cb9-898f-69d9b2f0537c/api" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.worker.apicaller connect.go:163 [a0a72c] "controller-0" successfully connected to "localhost:17070" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.apiserver.common password.go:75 setting mongo password for "controller-0" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.apiserver.common password.go:100 setting password for "controller-0" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.workers.modelworkermanager runner.go:592 start "a0a72c11-233c-4cb9-898f-69d9b2f0537c" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.workers.modelworkermanager runner.go:592 start "946658c9-f947-48cb-89e1-cfc55ab360ac" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/946658c9-f947-48cb-89e1-cfc55ab360ac/api" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.worker.migrationminion worker.go:142 migration phase is now: NONE migration 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.agent identity.go:28 removing system identity file 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.worker.logger logger.go:120 logger worker started 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 WARNING juju.worker.proxyupdater proxyupdater.go:241 unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 ERROR juju.worker.proxyupdater proxyupdater.go:279 error writing apt proxy config file: open /etc/apt/apt.conf.d/95-juju-proxy-settings: read-only file system 2024-04-16T08:07:02.679Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.api apiclient.go:707 connection established to "wss://localhost:17070/model/a0a72c11-233c-4cb9-898f-69d9b2f0537c/api" 2024-04-16T08:07:02.717Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.logforwarder logforwarder.go:105 config change - log forwarding not enabled 2024-04-16T08:07:02.769Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.logger logger.go:120 logger worker started 2024-04-16T08:07:02.795Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.pruner.statushistory worker.go:98 pruner config: max age: 336h0m0s, max collection size 5120M for controller (a0a72c11-233c-4cb9-898f-69d9b2f0537c) 2024-04-16T08:07:02.795Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.caasapplicationprovisioner.runner runner.go:592 start "controller" 2024-04-16T08:07:02.795Z [jujud] 2024-04-16 08:07:02 INFO juju.worker.pruner.action worker.go:98 pruner config: max age: 336h0m0s, max collection size 5120M for controller (a0a72c11-233c-4cb9-898f-69d9b2f0537c) 2024-04-16T08:07:02.919Z [jujud] 2024-04-16 08:07:02 INFO juju.kubernetes.provider operator.go:187 creating/updating kubeflow-volumes operator 2024-04-16T08:07:02.919Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:02 INFO juju.kubernetes.provider operator.go:187 creating/updating kubeflow-volumes operator 2024-04-16T08:07:03.905Z [jujud] 2024-04-16 08:07:03 INFO juju.kubernetes.klog klog.go:55 Waited for 1.108244048s due to client-side throttling, not priority and fairness, request: GET:https://10.152.183.1:443/api/v1/namespaces/test/services?labelSelector=app.kubernetes.io%2Fmanaged-by%3Djuju%2Capp.kubernetes.io%2Fname%3Dkubeflow-volumes 2024-04-16T08:07:03.905Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:03 INFO juju.kubernetes.klog klog.go:55 Waited for 1.108244048s due to client-side throttling, not priority and fairness, request: GET:https://10.152.183.1:443/api/v1/namespaces/test/services?labelSelector=app.kubernetes.io%2Fmanaged-by%3Djuju%2Capp.kubernetes.io%2Fname%3Dkubeflow-volumes 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:06:58 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [946658] "model-946658c9-f947-48cb-89e1-cfc55ab360ac" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:02 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [946658] "model-946658c9-f947-48cb-89e1-cfc55ab360ac" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":2,"response":{"results":[{"result":{"type":"kubernetes","name":"microk8s","region":"localhost","endpoint":"https://127.0.0.1:16443","credential":{"auth-type":"oauth2","attrs":{"Token":"eyJhbGciOiJSUzI1NiIsImtpZCI6Imw3aHRaNUhFVXo5ZzBfQm12X29udHlWMFhIM0hhV00wbGVTUFM4bjRoQzQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJqdWp1LWNyZWRlbnRpYWwtbWljcm9rOHMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoianVqdS1jcmVkZW50aWFsLW1pY3JvazhzIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYTU4OTE2ZGYtMjU4My00ZTRmLWJjNGUtMGU3YzRkOTc0Y2Q5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmp1anUtY3JlZGVudGlhbC1taWNyb2s4cyJ9.sHDWUtYgPK8yz5VxuOjrE7k41hVNi63izhk6g0JAs2oHC_zREjRHYnZNL9sk02UvR3t7-Wvp8WEdCClPbk1LmeQ9t6O17uvRosbhVII1hmev-F9WJwBsL8CNxpJfVccxTCzN1WcCIdcbN8MVPvqRM3-Vu7QzFc9asAF8MdjtPrSCZHEmj5g4mELCccMqn8BAUYl006jy8fLvUYp8tavt846H2hJtrOjUTAU1BCkJ8s5wCP1OeCid_BKhBssr6uDjFoocV6iOkU8U133xth6rgpJ8px65N8ELEuGclM2N3EOQPcg8-W6ML4RU1UYXpFyOyIcbrO-syiLbpiUt3XGqzA"}},"cacertificates":["-----BEGIN CERTIFICATE-----\nMIIDDzCCAfegAwIBAgIUCe6OKAo/xxNbpfmM2qgEdxIwoREwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMMTAuMTUyLjE4My4xMB4XDTI0MDQxNTE1NTAxMVoXDTM0\nMDQxMzE1NTAxMVowFzEVMBMGA1UEAwwMMTAuMTUyLjE4My4xMIIBIjANBgkqhkiG\n9w0BAQEFAAOCAQ8AMIIBCgKCAQEA7vTsKNvMxWYjoF+pTIJNNOxiEp55Ti+rpJ/W\nfNWcBEE7kHawEmHFt6YGbKuPQCjF/X/C+1SKTFZsvqVTijQY4IpOG8ATW9jQZgLC\nL3vqnwLO8w47oDvcHArHAshkXMLWhtZfAGLNe1AM7hTwQvzKEO1bATZ8J/X5nBkD\nzwO6hScf+WkHS56twz2EP8XbwtR5fK/1oeF7LrM+vcpWkPOH8H+fZSgeYbbTbDrQ\nQosBSl3l48ZLCAcSIFMbvlcGlPImHjFoIYaqp6ePEQDE5S4nfSbR3uhWZnjBW9iH\nhmv9RetHoWlOIqCqF5D6uHXz9afZCSC/V/mlC148QNHa4Sqk7QIDAQABo1MwUTAd\nBgNVHQ4EFgQUeSNuZ6swiZZf7Ljqig5uCJPqLcwwHwYDVR0jBBgwFoAUeSNuZ6sw\niZZf7Ljqig5uCJPqLcwwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOC\nAQEAg5IEmUkC67+wKdtMK/GHuf266+kkTcTYYSVz/xIjWKWvDy5RaUeI7gYEB9gp\n100XfzeLM3eHXCt3rUTZcisBXk5af+I8G+jkQLL6eL+g+NRi/92Z1zvCIyuRMzhF\nunTA809YI3+bFWZOWfm4BTfxXsIcpehTV9FN3t1Vw047pMEnvNLPs7i+OqLYKnhV\n+Hf5T/+p0CfJ0rrOM4ARM3oHtXow/1lgmTo4BoaGCL0WlPUC7UUza7ea74dx9lck\nr3Twd+L8KeaKD1Wm7QK7lqjYNlv1FkfvgJsgelC7hJPXlMfKDjVW/ZFo4UMMXsxb\nYQWAUP9sc+81xx3TAvjw3owdjg==\n-----END CERTIFICATE-----\n"],"is-controller-cloud":true}}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":5,"type":"CAASAgent","version":2,"request":"ModelConfig","params":{}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":5,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":6,"type":"CAASAgent","version":2,"request":"ControllerConfig","params":{}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":6,"response":{"config":{"agent-logfile-max-backups":2,"agent-logfile-max-size":"100M","api-port":17070,"api-port-open-delay":"2s","audit-log-capture-args":false,"audit-log-exclude-methods":["ReadOnlyMethods"],"audit-log-max-backups":10,"audit-log-max-size":"300M","auditing-enabled":true,"ca-cert":"-----BEGIN CERTIFICATE-----\nMIIEEjCCAnqgAwIBAgIUFFlW4Nf1n4+uKOuxAge1YVG8yVwwDQYJKoZIhvcNAQEL\nBQAwITENMAsGA1UEChMESnVqdTEQMA4GA1UEAxMHanVqdS1jYTAeFw0yNDA0MTUx\nNTUyMDlaFw0zNDA0MTUxNTU3MDlaMCExDTALBgNVBAoTBEp1anUxEDAOBgNVBAMT\nB2p1anUtY2EwggGiMA0GCSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDL8U01OYzW\n8jPkvEO8JYO/yPsdBBBVk5myLgrXcuL78FRDLbhyZCbjEgrMX4BLPyZEQ8RASWrd\nFx1rGlVCAa4PH3oxTAWdzVn6HRyIK77hXXVGtEQDD4lIMuoMFHnODSoe1YbSys9n\noK2bYkIO1sP1UGv04GNuVgtBa9r+Ss4rpPuj3S1Si5CsPTSB4ht16/dDpGpqFN5t\nGGgJxudE3hIeXuYQJlZgr2kRmkP3ik7tmVEX29qvupVB4Jo64c91WLYtDToozrPS\n339wW3VO8rUtrFWdWKCSY/SYhY33DZ3So6JexvQeQAP8nsFIfFAJM/d+a3EOkicL\nlPT3YGJ+gBqNrEsXL7K+dW5KK4MpZ+RONmECLnW/KZ7DLurEErh8GWXSiogmmA35\nrtMIxDAaR4wvVccE4chpavapgWEIQ7t0qJTenZ+Oij4okJN5yPwQ+wQuRYZqrXe4\n1Rsdm8fXwN1gHKEd6VhVmRm3wNNgXUoT7T0BW7pH9AOprAcSd0G5ZtcCAwEAAaNC\nMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFLLo\njd/5Eh3QhRt8K+Y1r46B6g+kMA0GCSqGSIb3DQEBCwUAA4IBgQAJGxbR+w1eJkz8\nxCpVnKJZoG5PemzBKJyinAu4mjmcdJVaZ20ovKUpe69juWxJ94MWz8IVlw6/6qUR\nyIKonq4RIQ+gBxtVp69xuMvbz2JPqT3OcLwQenV7tJmurOMyFtjHZZfEpYOTZoNo\nURHOA973kvKLewIPG1ntIncYqu/jImZOEOZ9/3frdM2OFahfWnJNyxZ7Cv5Lxy4+\nUUFiHwR1uXMtFte+ZIItRhKIWTosKyIU7Icq3dVsGZHTSyYrsIpd6NydvoUEXqRA\nN1/ftJpMNDyNqrZDkghoZRtN1DEsrEmgJMnuK1EzHatBdxqMnC/xxtTSvfV1OGkT\nkZ68qkgvZYSb2mbJdt52X85n40UfmpaGRi7SgFrX0YDEr5OoaS+XJY6wakMr6Rn7\ngOIHmPABEQX7EZfmPvSWHgHRKPzC+c/DtFtbpQv2Oe/Ug43LJwnvX7N5Eq8JzgVb\ngp/wBJUMtkCvDsswsSzQ1lmcMGEmbjqZhmJ9VzV/mY4+Ja9VVsg=\n-----END CERTIFICATE-----\n","controller-name":"uk8s","controller-uuid":"72c875e9-49dd-4af5-8ea4-deb43c2d37c4","juju-db-snap-channel":"4.4/stable","max-agent-state-size":524288,"max-charm-state-size":2097152,"max-debug-log-duration":"24h0m0s","max-prune-txn-batch-size":1000000,"max-prune-txn-passes":100,"max-txn-log-size":"10M","metering-url":"https://api.jujucharms.com/omnibus/v3","migration-agent-wait-time":"15m0s","model-logfile-max-backups":2,"model-logfile-max-size":"10M","model-logs-size":"20M","mongo-memory-profile":"default","prune-txn-query-count":1000,"prune-txn-sleep-time":"10ms","query-tracing-enabled":false,"query-tracing-threshold":"1s","set-numa-control-policy":false,"state-port":37017}}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.kubernetes.provider provider.go:164 opening model "test". 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.kubernetes.provider provider.go:129 using in-cluster config 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":7,"type":"Logger","version":1,"request":"WatchLoggingConfig","params":{"entities":[{"tag":"model-946658c9-f947-48cb-89e1-cfc55ab360ac"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":7,"response":{"results":[{"NotifyWatcherId":"1"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":8,"type":"Logger","version":1,"request":"LoggingConfig","params":{"entities":[{"tag":"model-946658c9-f947-48cb-89e1-cfc55ab360ac"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":9,"type":"NotifyWatcher","version":1,"id":"1","request":"Next","params":{}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":8,"response":{"results":[{"result":"\u003croot\u003e=TRACE;unit=TRACE"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":3,"response":{"results":[{"NotifyWatcherId":"2"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":10,"type":"NotifyWatcher","version":1,"id":"2","request":"Next","params":{}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caasupgrader upgrader.go:128 current agent binary version: 3.4.2 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":11,"type":"Upgrader","version":1,"request":"DesiredVersion","params":{"entities":[{"tag":"model-946658c9-f947-48cb-89e1-cfc55ab360ac"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":11,"response":{"results":[{"version":"3.4.2"}]}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:529 running "caas-broker-tracker" manifold worker 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:540 registered "caas-broker-tracker" manifold worker 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "caas-broker-tracker" manifold worker started at 2024-04-16 08:07:06.264122658 +0000 UTC 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:756 restarting dependents of "caas-broker-tracker" manifold 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":12,"type":"CAASAgent","version":2,"request":"WatchForModelConfigChanges","params":{}} 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:511 starting "caas-admission" manifold worker in 11ms... 2024-04-16T08:07:06.265Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:511 starting "caas-rbac-mapper" manifold worker in 9ms... 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:519 starting "caas-rbac-mapper" manifold worker 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-rbac-mapper" manifold requested "caas-broker-tracker" resource 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:529 running "caas-rbac-mapper" manifold worker 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:540 registered "caas-rbac-mapper" manifold worker 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "caas-rbac-mapper" manifold worker started at 2024-04-16 08:07:06.274488061 +0000 UTC 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:756 restarting dependents of "caas-rbac-mapper" manifold 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:607 "caas-admission" manifold worker bounced while starting 2024-04-16T08:07:06.274Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:511 starting "caas-admission" manifold worker in 9ms... 2024-04-16T08:07:06.275Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":12,"response":{"NotifyWatcherId":"3"}} 2024-04-16T08:07:06.275Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":13,"type":"CAASAgent","version":2,"request":"WatchCloudSpecsChanges","params":{"entities":[{"tag":"model-946658c9-f947-48cb-89e1-cfc55ab360ac"}]}} 2024-04-16T08:07:06.275Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":14,"type":"NotifyWatcher","version":1,"id":"3","request":"Next","params":{}} 2024-04-16T08:07:06.284Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:519 starting "caas-admission" manifold worker 2024-04-16T08:07:06.284Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "agent" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "certificate-watcher" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "caas-broker-tracker" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "caas-rbac-mapper" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "model-http-server" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "caas-admission" manifold requested "model-http-server" resource 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:529 running "caas-admission" manifold worker 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:540 registered "caas-admission" manifold worker 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "caas-admission" manifold worker started at 2024-04-16 08:07:06.28483943 +0000 UTC 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:756 restarting dependents of "caas-admission" manifold 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caasadmission controller.go:66 installing caas admission handler at /k8s/admission/946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16T08:07:06.285Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 INFO juju.worker.caasadmission controller.go:72 ensuring model k8s webhook configurations 2024-04-16T08:07:06.287Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":13,"response":{"results":[{"NotifyWatcherId":"4"}]}} 2024-04-16T08:07:06.287Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caas broker.go:152 waiting for config and credential notifications 2024-04-16T08:07:06.287Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caas broker.go:160 reloading model config 2024-04-16T08:07:06.287Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":15,"type":"CAASAgent","version":2,"request":"ModelConfig","params":{}} 2024-04-16T08:07:06.287Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":16,"type":"NotifyWatcher","version":1,"id":"4","request":"Next","params":{}} 2024-04-16T08:07:06.288Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":15,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:06.288Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caas broker.go:152 waiting for config and credential notifications 2024-04-16T08:07:06.288Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":17,"type":"CAASAgent","version":2,"request":"CloudSpec","params":{"entities":[{"tag":"model-946658c9-f947-48cb-89e1-cfc55ab360ac"}]}} 2024-04-16T08:07:06.289Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":17,"response":{"results":[{"result":{"type":"kubernetes","name":"microk8s","region":"localhost","endpoint":"https://127.0.0.1:16443","credential":{"auth-type":"oauth2","attrs":{"Token":"eyJhbGciOiJSUzI1NiIsImtpZCI6Imw3aHRaNUhFVXo5ZzBfQm12X29udHlWMFhIM0hhV00wbGVTUFM4bjRoQzQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJqdWp1LWNyZWRlbnRpYWwtbWljcm9rOHMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoianVqdS1jcmVkZW50aWFsLW1pY3JvazhzIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYTU4OTE2ZGYtMjU4My00ZTRmLWJjNGUtMGU3YzRkOTc0Y2Q5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmp1anUtY3JlZGVudGlhbC1taWNyb2s4cyJ9.sHDWUtYgPK8yz5VxuOjrE7k41hVNi63izhk6g0JAs2oHC_zREjRHYnZNL9sk02UvR3t7-Wvp8WEdCClPbk1LmeQ9t6O17uvRosbhVII1hmev-F9WJwBsL8CNxpJfVccxTCzN1WcCIdcbN8MVPvqRM3-Vu7QzFc9asAF8MdjtPrSCZHEmj5g4mELCccMqn8BAUYl006jy8fLvUYp8tavt846H2hJtrOjUTAU1BCkJ8s5wCP1OeCid_BKhBssr6uDjFoocV6iOkU8U133xth6rgpJ8px65N8ELEuGclM2N3EOQPcg8-W6ML4RU1UYXpFyOyIcbrO-syiLbpiUt3XGqzA"}},"cacertificates":["-----BEGIN CERTIFICATE-----\nMIIDDzCCAfegAwIBAgIUCe6OKAo/xxNbpfmM2qgEdxIwoREwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMMTAuMTUyLjE4My4xMB4XDTI0MDQxNTE1NTAxMVoXDTM0\nMDQxMzE1NTAxMVowFzEVMBMGA1UEAwwMMTAuMTUyLjE4My4xMIIBIjANBgkqhkiG\n9w0BAQEFAAOCAQ8AMIIBCgKCAQEA7vTsKNvMxWYjoF+pTIJNNOxiEp55Ti+rpJ/W\nfNWcBEE7kHawEmHFt6YGbKuPQCjF/X/C+1SKTFZsvqVTijQY4IpOG8ATW9jQZgLC\nL3vqnwLO8w47oDvcHArHAshkXMLWhtZfAGLNe1AM7hTwQvzKEO1bATZ8J/X5nBkD\nzwO6hScf+WkHS56twz2EP8XbwtR5fK/1oeF7LrM+vcpWkPOH8H+fZSgeYbbTbDrQ\nQosBSl3l48ZLCAcSIFMbvlcGlPImHjFoIYaqp6ePEQDE5S4nfSbR3uhWZnjBW9iH\nhmv9RetHoWlOIqCqF5D6uHXz9afZCSC/V/mlC148QNHa4Sqk7QIDAQABo1MwUTAd\nBgNVHQ4EFgQUeSNuZ6swiZZf7Ljqig5uCJPqLcwwHwYDVR0jBBgwFoAUeSNuZ6sw\niZZf7Ljqig5uCJPqLcwwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOC\nAQEAg5IEmUkC67+wKdtMK/GHuf266+kkTcTYYSVz/xIjWKWvDy5RaUeI7gYEB9gp\n100XfzeLM3eHXCt3rUTZcisBXk5af+I8G+jkQLL6eL+g+NRi/92Z1zvCIyuRMzhF\nunTA809YI3+bFWZOWfm4BTfxXsIcpehTV9FN3t1Vw047pMEnvNLPs7i+OqLYKnhV\n+Hf5T/+p0CfJ0rrOM4ARM3oHtXow/1lgmTo4BoaGCL0WlPUC7UUza7ea74dx9lck\nr3Twd+L8KeaKD1Wm7QK7lqjYNlv1FkfvgJsgelC7hJPXlMfKDjVW/ZFo4UMMXsxb\nYQWAUP9sc+81xx3TAvjw3owdjg==\n-----END CERTIFICATE-----\n"],"is-controller-cloud":true}}]}} 2024-04-16T08:07:06.289Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.caas broker.go:152 waiting for config and credential notifications 2024-04-16T08:07:06.343Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: unit-controller-0 2024-04-16 08:07:06 INFO juju.worker.uniter.operation runhook.go:186 ran "api-server-pebble-ready" hook (via hook dispatching script: dispatch) 2024-04-16T08:07:06.705Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.muxhttpserver sni.go:23 received tls client hello for server name modeloperator.test.svc 2024-04-16T08:07:06.705Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.worker.muxhttpserver sni.go:49 no matching certificate found for tls client hello modeloperator.test.svc, using default certificate 2024-04-16T08:07:06.760Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:06 DEBUG juju.kubernetes.provider admissionregistration.go:154 updating MutatingWebhookConfiguration "juju-model-admission-test" 2024-04-16T08:07:06.911Z [jujud] 2024-04-16 08:07:06 INFO juju.apiserver.connection request_notifier.go:96 agent login: application-kubeflow-volumes for 946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16T08:07:06.911Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:06 INFO juju.apiserver.connection request_notifier.go:96 agent login: application-kubeflow-volumes for 946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 INFO juju.cmd supercommand.go:56 running jujud [3.4.2 a80becbb4da5985fa53c63824a4bd809e9d03954 gc go1.20.14] 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.cmd supercommand.go:57 args: []string{"/var/lib/juju/tools/jujud", "caasoperator", "--application-name=kubeflow-volumes", "--debug"} 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.agent agent.go:600 read agent config, format "2.0" 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 INFO juju.worker.upgradesteps worker.go:60 upgrade steps for 3.4.2 have already been run. 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 INFO juju.cmd.jujud caasoperator.go:205 caas operator application-kubeflow-volumes start (3.4.2 [gc]) 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.cmd.jujud runner.go:416 start "api" 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 INFO juju.cmd.jujud runner.go:592 start "api" 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "upgrade-steps-gate" manifold worker started at 2024-04-16 08:06:56.579040919 +0000 UTC 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "caas-units-manager" manifold worker started at 2024-04-16 08:06:56.579176839 +0000 UTC 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "upgrade-steps-flag" manifold worker started at 2024-04-16 08:06:56.579200478 +0000 UTC 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "agent" manifold worker started at 2024-04-16 08:06:56.579471569 +0000 UTC 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "clock" manifold worker started at 2024-04-16 08:06:56.579619297 +0000 UTC 2024-04-16T08:07:06.939Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:603 "caas-units-manager" manifold worker completed successfully 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "migration-fortress" manifold worker started at 2024-04-16 08:06:56.579833653 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.introspection worker.go:127 introspection worker listening on "@jujud-application-kubeflow-volumes" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.introspection worker.go:152 stats worker now serving 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.cmd.jujud runner.go:424 "api" started 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "caas-units-manager" manifold worker started at 2024-04-16 08:06:56.589377664 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.apicaller connect.go:129 connecting with old password 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:580 "api-config-watcher" manifold worker started at 2024-04-16 08:06:56.590657829 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.api apiclient.go:825 looked up controller-service.controller-uk8s.svc.cluster.local -> [10.152.183.39] 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.apicaller connect.go:160 [946658] failed to connect 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.dependency engine.go:618 "api-caller" manifold worker stopped: [946658] "application-kubeflow-volumes" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] stack trace: 2024-04-16T08:07:06.940Z [jujud] dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.gorillaDialWebsocket:756: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial1:1176: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial:1151: unable to connect to API 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialWebsocketMulti:1047: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialAPI:705: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.Open:218: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.connectFallback:161: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.OnlyConnect:58: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.ManifoldConfig.startFunc.func1:97: [946658] "application-kubeflow-volumes" cannot open api 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.worker.apicaller connect.go:129 connecting with old password 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:56 DEBUG juju.api apiclient.go:825 looked up controller-service.controller-uk8s.svc.cluster.local -> [10.152.183.39] 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:57 DEBUG juju.worker.apicaller connect.go:160 [946658] failed to connect 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:57 DEBUG juju.worker.dependency engine.go:618 "api-caller" manifold worker stopped: [946658] "application-kubeflow-volumes" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] stack trace: 2024-04-16T08:07:06.940Z [jujud] dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.gorillaDialWebsocket:756: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial1:1176: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial:1151: unable to connect to API 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialWebsocketMulti:1047: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialAPI:705: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.Open:218: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.connectFallback:161: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.OnlyConnect:58: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.ManifoldConfig.startFunc.func1:97: [946658] "application-kubeflow-volumes" cannot open api 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:06:57 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [946658] "application-kubeflow-volumes" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:01 DEBUG juju.worker.apicaller connect.go:129 connecting with old password 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:01 DEBUG juju.api apiclient.go:825 looked up controller-service.controller-uk8s.svc.cluster.local -> [10.152.183.39] 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:01 DEBUG juju.worker.apicaller connect.go:160 [946658] failed to connect 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:01 DEBUG juju.worker.dependency engine.go:618 "api-caller" manifold worker stopped: [946658] "application-kubeflow-volumes" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] stack trace: 2024-04-16T08:07:06.940Z [jujud] dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.gorillaDialWebsocket:756: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial1:1176: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialer.dial:1151: unable to connect to API 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialWebsocketMulti:1047: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.dialAPI:705: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/api.Open:218: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.connectFallback:161: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.OnlyConnect:58: 2024-04-16T08:07:06.940Z [jujud] github.com/juju/juju/worker/apicaller.ManifoldConfig.startFunc.func1:97: [946658] "application-kubeflow-volumes" cannot open api 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:01 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [946658] "application-kubeflow-volumes" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.apicaller connect.go:129 connecting with old password 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.api apiclient.go:1172 successfully dialed "wss://10.152.183.39:17070/model/946658c9-f947-48cb-89e1-cfc55ab360ac/api" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.api apiclient.go:1074 cannot resolve "controller-service.controller-uk8s.svc.cluster.local": lookup controller-service.controller-uk8s.svc.cluster.local: operation was canceled 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.api apiclient.go:707 connection established to "wss://10.152.183.39:17070/model/946658c9-f947-48cb-89e1-cfc55ab360ac/api" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.apicaller connect.go:163 [946658] "application-kubeflow-volumes" successfully connected to "10.152.183.39:17070" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "api-caller" manifold worker started at 2024-04-16 08:07:06.914842957 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:603 "caas-units-manager" manifold worker completed successfully 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "caas-units-manager" manifold worker started at 2024-04-16 08:07:06.923966837 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "upgrader" manifold worker started at 2024-04-16 08:07:06.924237261 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "log-sender" manifold worker started at 2024-04-16 08:07:06.926101141 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "upgrade-steps-runner" manifold worker started at 2024-04-16 08:07:06.926191101 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:603 "upgrade-steps-runner" manifold worker completed successfully 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "migration-minion" manifold worker started at 2024-04-16 08:07:06.92636684 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "migration-inactive-flag" manifold worker started at 2024-04-16 08:07:06.928216512 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.caasupgrader upgrader.go:128 current agent binary version: 3.4.2 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.migrationminion worker.go:142 migration phase is now: NONE migration 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "charm-dir" manifold worker started at 2024-04-16 08:07:06.937466523 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "proxy-config-updater" manifold worker started at 2024-04-16 08:07:06.937492886 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "secret-drain-worker" manifold worker started at 2024-04-16 08:07:06.93749929 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.logger logger.go:65 initial log config: "=DEBUG" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "api-address-updater" manifold worker started at 2024-04-16 08:07:06.939291199 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.logger logger.go:120 logger worker started 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "logging-config-updater" manifold worker started at 2024-04-16 08:07:06.939311118 +0000 UTC 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.proxyupdater proxyupdater.go:135 applying in-process legacy proxy settings proxy.Settings{Http:"", Https:"", Ftp:"", NoProxy:"10.152.183.39,127.0.0.1,::1,controller-service.controller-uk8s.svc.cluster.local,localhost", AutoNoProxy:""} 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.logger logger.go:93 reconfiguring logging from "=DEBUG" to "=TRACE;unit=TRACE" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 WARNING juju.worker.proxyupdater proxyupdater.go:241 unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.proxyupdater proxyupdater.go:271 new apt proxy settings proxy.Settings{Http:"", Https:"", Ftp:"", NoProxy:"127.0.0.1,::1,localhost", AutoNoProxy:""} 2024-04-16T08:07:06.940Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":17,"type":"ProxyUpdater","version":2,"request":"WatchForProxyConfigAndAPIHostPortChanges","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.946Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":14,"response":{"results":[{"phase":"NONE"}]}} 2024-04-16T08:07:06.946Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":15,"response":{"results":[{"result":{"should-retry":true,"min-retry-time":5000000000,"max-retry-time":300000000000,"jitter-retry-time":true,"retry-time-factor":2}}]}} 2024-04-16T08:07:06.946Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:529 running "hook-retry-strategy" manifold worker 2024-04-16T08:07:06.946Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:540 registered "hook-retry-strategy" manifold worker 2024-04-16T08:07:06.947Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":18,"type":"RetryStrategy","version":1,"request":"WatchRetryStrategy","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.947Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "hook-retry-strategy" manifold worker started at 2024-04-16 08:07:06.946953466 +0000 UTC 2024-04-16T08:07:06.947Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:756 restarting dependents of "hook-retry-strategy" manifold 2024-04-16T08:07:06.947Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:607 "operator" manifold worker bounced while starting 2024-04-16T08:07:06.947Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:511 starting "operator" manifold worker in 10ms... 2024-04-16T08:07:06.948Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":11,"response":{"NotifyWatcherId":"4"}} 2024-04-16T08:07:06.948Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.secretsdrainworker worker.go:110 got new secret backend 2024-04-16T08:07:06.948Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":19,"type":"SecretsDrain","version":1,"request":"GetSecretsToDrain","params":{}} 2024-04-16T08:07:06.948Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":20,"type":"NotifyWatcher","version":1,"id":"4","request":"Next","params":{}} 2024-04-16T08:07:06.949Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":19,"response":{"results":null}} 2024-04-16T08:07:06.949Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.secretsdrainworker worker.go:117 no secrets to drain 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":13,"response":{"NotifyWatcherId":"5"}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":17,"response":{"results":[{"NotifyWatcherId":"6"}]}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":21,"type":"CAASOperator","version":1,"request":"APIHostPorts","params":{}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:519 starting "operator" manifold worker 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":22,"type":"ProxyUpdater","version":2,"request":"ProxyConfig","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "migration-inactive-flag" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":23,"type":"NotifyWatcher","version":1,"id":"5","request":"Next","params":{}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":18,"response":{"results":[{"NotifyWatcherId":"7"}]}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":24,"type":"NotifyWatcher","version":1,"id":"6","request":"Next","params":{}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":25,"type":"RetryStrategy","version":1,"request":"RetryStrategy","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "migration-fortress" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "agent" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "api-caller" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "clock" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":26,"type":"CAASOperator","version":1,"request":"CurrentModel","params":{}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":27,"type":"NotifyWatcher","version":1,"id":"7","request":"Next","params":{}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":26,"response":{"name":"test","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","type":"caas"}} 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "charm-dir" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency context.go:46 "operator" manifold requested "hook-retry-strategy" resource 2024-04-16T08:07:06.957Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":25,"response":{"results":[{"result":{"should-retry":true,"min-retry-time":5000000000,"max-retry-time":300000000000,"jitter-retry-time":true,"retry-time-factor":2}}]}} 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":21,"response":{"servers":[[{"value":"controller-service.controller-uk8s.svc.cluster.local","type":"hostname","scope":"local-cloud","port":17070},{"value":"10.152.183.39","type":"ipv4","scope":"local-cloud","port":17070}]]}} 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:144 cannot get "lxdbr0" addresses: route ip+net: no such network interface (ignoring) 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:144 cannot get "virbr0" addresses: route ip+net: no such network interface (ignoring) 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:122 not filtering invalid IP: "controller-service.controller-uk8s.svc.cluster.local" 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:132 including address local-cloud:controller-service.controller-uk8s.svc.cluster.local for machine 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:132 including address local-cloud:10.152.183.39 for machine 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.network network.go:160 addresses after filtering: [local-cloud:controller-service.controller-uk8s.svc.cluster.local local-cloud:10.152.183.39] 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.apiaddressupdater apiaddressupdater.go:98 updating API hostPorts to [[controller-service.controller-uk8s.svc.cluster.local:17070 10.152.183.39:17070]] 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":22,"response":{"results":[{"legacy-proxy-settings":{"http":"","https":"","ftp":"","no-proxy":"10.152.183.39,127.0.0.1,::1,controller-service.controller-uk8s.svc.cluster.local,localhost"},"juju-proxy-settings":{"http":"","https":"","ftp":"","no-proxy":"127.0.0.1,::1,localhost"},"apt-proxy-settings":{"http":"","https":"","ftp":"","no-proxy":"127.0.0.1,::1,localhost"},"snap-proxy-settings":{"http":"","https":"","ftp":"","no-proxy":""}}]}} 2024-04-16T08:07:06.958Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.proxyupdater proxyupdater.go:135 applying in-process legacy proxy settings proxy.Settings{Http:"", Https:"", Ftp:"", NoProxy:"10.152.183.39,127.0.0.1,::1,controller-service.controller-uk8s.svc.cluster.local,localhost", AutoNoProxy:""} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":28,"type":"Logger","version":1,"request":"WatchLoggingConfig","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.agent agent.go:651 API server address details [["controller-service.controller-uk8s.svc.cluster.local:17070" "10.152.183.39:17070"]] written to agent config as ["10.152.183.39:17070" "controller-service.controller-uk8s.svc.cluster.local:17070"] 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":28,"response":{"results":[{"NotifyWatcherId":"8"}]}} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":29,"type":"Logger","version":1,"request":"LoggingConfig","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":30,"type":"NotifyWatcher","version":1,"id":"8","request":"Next","params":{}} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":29,"response":{"results":[{"result":"\u003croot\u003e=TRACE;unit=TRACE"}]}} 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:529 running "operator" manifold worker 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:540 registered "operator" manifold worker 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 DEBUG juju.worker.dependency engine.go:580 "operator" manifold worker started at 2024-04-16 08:07:06.981826547 +0000 UTC 2024-04-16T08:07:06.981Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.worker.dependency engine.go:756 restarting dependents of "operator" manifold 2024-04-16T08:07:06.982Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":31,"type":"CAASOperator","version":1,"request":"Charm","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:06.983Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":31,"response":{"results":[{"result":{"url":"ch:amd64/focal/kubeflow-volumes-289","sha256":"e7404046f5d59a0cc8fb2de6a7f1d1fabb016466bab76df228605516e33ef6a7","charm-modified-version":0,"deployment-mode":"workload"}}]}} 2024-04-16T08:07:06.983Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":32,"type":"CAASOperator","version":1,"request":"SetStatus","params":{"entities":[{"tag":"application-kubeflow-volumes","status":"maintenance","info":"downloading charm (ch:amd64/focal/kubeflow-volumes-289)","data":null}]}} 2024-04-16T08:07:06.989Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":32,"response":{"results":[{}]}} 2024-04-16T08:07:06.989Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.worker.caasoperator.charm bundles.go:82 downloading ch:amd64/focal/kubeflow-volumes-289 from API server 2024-04-16T08:07:06.989Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:06 INFO juju.downloader download.go:112 downloading from ch:amd64/focal/kubeflow-volumes-289 2024-04-16T08:07:07.046Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:07 INFO juju.downloader download.go:95 download complete ("ch:amd64/focal/kubeflow-volumes-289") 2024-04-16T08:07:07.065Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:07 INFO juju.downloader download.go:175 download verified ("ch:amd64/focal/kubeflow-volumes-289") 2024-04-16T08:07:07.074Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:07 DEBUG juju.worker.caasoperator.charm manifest_deployer.go:143 preparing to deploy charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:07.075Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:07 DEBUG juju.worker.caasoperator.charm manifest_deployer.go:132 deploying charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:07.307Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:07 DEBUG juju.worker.muxhttpserver sni.go:23 received tls client hello for server name modeloperator.test.svc 2024-04-16T08:07:07.319Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:07 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:07.324Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:07 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:07.332Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:07 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:07.707Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:07 DEBUG juju.worker.caasadmission handler.go:113 received admission request for model-exec of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:08.107Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:08.307Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:08.313Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of rbac.authorization.k8s.io/v1, Kind=Role in namespace test 2024-04-16T08:07:08.324Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of rbac.authorization.k8s.io/v1, Kind=Role in namespace test 2024-04-16T08:07:08.339Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of rbac.authorization.k8s.io/v1, Kind=RoleBinding in namespace test 2024-04-16T08:07:08.906Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:08 DEBUG juju.worker.caasadmission handler.go:113 received admission request for modeloperator of /v1, Kind=Service in namespace test 2024-04-16T08:07:09.018Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: model-a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16 08:06:58 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [a0a72c] "model-a0a72c11-233c-4cb9-898f-69d9b2f0537c" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:09.018Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: model-a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16 08:07:03 ERROR juju.worker.dependency engine.go:695 "api-caller" manifold worker returned unexpected error: [a0a72c] "model-a0a72c11-233c-4cb9-898f-69d9b2f0537c" cannot open api: unable to connect to API: dial tcp 10.152.183.39:17070: connect: connection refused 2024-04-16T08:07:09.021Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: model-a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16 08:07:09 INFO juju.worker.caasupgrader upgrader.go:113 abort check blocked until version event received 2024-04-16T08:07:09.021Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: model-a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16 08:07:09 INFO juju.worker.caasupgrader upgrader.go:119 unblocking abort check 2024-04-16T08:07:09.046Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: model-a0a72c11-233c-4cb9-898f-69d9b2f0537c 2024-04-16 08:07:09 INFO juju.worker.caasadmission controller.go:72 ensuring model k8s webhook configurations 2024-04-16T08:07:09.105Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:09 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:09.109Z [jujud] 2024-04-16 08:07:09 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:09.110Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:09 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:10.343Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:10 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of rbac.authorization.k8s.io/v1, Kind=RoleBinding in namespace test 2024-04-16T08:07:10.355Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:10 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator-config of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:10.364Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:10 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of apps/v1, Kind=StatefulSet in namespace test 2024-04-16T08:07:10.372Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:10 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-operator of apps/v1, Kind=StatefulSet in namespace test 2024-04-16T08:07:10.642Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.charm manifest_deployer.go:164 finishing deploy of charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:10.643Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator caasoperator.go:396 starting caas operator juju-exec listener on &{tcp :30666 0xc000b42180} 2024-04-16T08:07:10.643Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.caasoperator caasoperator.go:430 operator "kubeflow-volumes" started 2024-04-16T08:07:10.643Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.uniter runlistener.go:117 juju-exec listener running 2024-04-16T08:07:10.643Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":33,"type":"CAASOperator","version":1,"request":"SetTools","params":{"agent-tools":[{"tag":"application-kubeflow-volumes","tools":{"version":"3.4.2-ubuntu-amd64"}}]}} 2024-04-16T08:07:10.648Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":33,"response":{"results":[{}]}} 2024-04-16T08:07:10.648Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":34,"type":"CAASOperator","version":1,"request":"WatchUnits","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.651Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":34,"response":{"results":[{"watcher-id":"9","changes":["kubeflow-volumes/2"]}]}} 2024-04-16T08:07:10.651Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":35,"type":"CAASOperator","version":1,"request":"WatchContainerStart","params":{"args":[{"entity":{"tag":"application-kubeflow-volumes"},"container":"(?:juju-pod-init|)"}]}} 2024-04-16T08:07:10.651Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":36,"type":"StringsWatcher","version":1,"id":"9","request":"Next","params":{}} 2024-04-16T08:07:10.656Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":35,"response":{"results":[{"watcher-id":"10"}]}} 2024-04-16T08:07:10.656Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":37,"type":"CAASOperator","version":1,"request":"SetStatus","params":{"entities":[{"tag":"application-kubeflow-volumes","status":"active","info":"","data":null}]}} 2024-04-16T08:07:10.656Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":38,"type":"StringsWatcher","version":1,"id":"10","request":"Next","params":{}} 2024-04-16T08:07:10.667Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":37,"response":{"results":[{}]}} 2024-04-16T08:07:10.667Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":39,"type":"CAASOperator","version":1,"request":"Watch","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":39,"response":{"results":[{"NotifyWatcherId":"11"}]}} 2024-04-16T08:07:10.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.remotestate watcher.go:139 got application change 2024-04-16T08:07:10.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":40,"type":"NotifyWatcher","version":1,"id":"11","request":"Next","params":{}} 2024-04-16T08:07:10.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":41,"type":"CAASOperator","version":1,"request":"Charm","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.679Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":41,"response":{"results":[{"result":{"url":"ch:amd64/focal/kubeflow-volumes-289","sha256":"e7404046f5d59a0cc8fb2de6a7f1d1fabb016466bab76df228605516e33ef6a7","charm-modified-version":0,"deployment-mode":"workload"}}]}} 2024-04-16T08:07:10.679Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":42,"type":"CAASOperator","version":1,"request":"Life","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.680Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":42,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:10.680Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator caasoperator.go:554 got unit change "kubeflow-volumes/2" (alive) 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 uniter.go:255 starting uniter for "kubeflow-volumes/2" 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.runner runner.go:416 start "kubeflow-volumes/2" 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.caasoperator.runner runner.go:592 start "kubeflow-volumes/2" 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.runner runner.go:424 "kubeflow-volumes/2" started 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.leadership tracker.go:125 kubeflow-volumes/2 making initial claim for kubeflow-volumes leadership 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":43,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:07:10.681Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":44,"type":"Uniter","version":19,"request":"Refresh","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.682Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":44,"response":{"Results":[{"Life":"alive","Resolved":"","Error":null}]}} 2024-04-16T08:07:10.682Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":45,"type":"Uniter","version":19,"request":"UnitStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.683Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":45,"response":{"results":[{"id":"","life":"","status":"active","info":"","data":{},"since":"2024-04-16T07:06:37.543921398Z"}]}} 2024-04-16T08:07:10.683Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.agent.tools symlinks.go:20 ensure jujuc symlinks in /var/lib/juju/tools/unit-kubeflow-volumes-2 2024-04-16T08:07:10.683Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.agent.tools symlinks.go:47 jujuc not found at /var/lib/juju/tools/unit-kubeflow-volumes-2/jujuc using jujud path /var/lib/juju/tools/unit-kubeflow-volumes-2/jujud 2024-04-16T08:07:10.683Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.agent.tools symlinks.go:50 target tools path /var/lib/juju/tools/unit-kubeflow-volumes-2/jujud 2024-04-16T08:07:10.683Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":46,"type":"Uniter","version":19,"request":"GetPrincipal","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.684Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":46,"response":{"results":[{"result":"","ok":false}]}} 2024-04-16T08:07:10.684Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":47,"type":"Uniter","version":19,"request":"State","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.684Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":43,"response":{"results":[{}]}} 2024-04-16T08:07:10.684Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.leadership tracker.go:194 kubeflow-volumes/2 promoted to leadership of kubeflow-volumes 2024-04-16T08:07:10.684Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:07:40.681159662 +0000 UTC m=+44.124475013 2024-04-16T08:07:10.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:07:25.681159662 +0000 UTC m=+29.124475013 2024-04-16T08:07:10.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":47,"response":{"results":[{"uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:10.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statemanager.go:129 initialising state manager: map[int]string{} 2024-04-16T08:07:10.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":48,"type":"Uniter","version":19,"request":"RelationsStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":48,"response":{"results":[{"results":null}]}} 2024-04-16T08:07:10.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:132 initialising relation state tracker: map[int]relation.State{ 2024-04-16T08:07:10.686Z [jujud] } 2024-04-16T08:07:10.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":49,"type":"Uniter","version":19,"request":"UnitStorageAttachments","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":49,"response":{"results":[{"result":{"ids":[]}}]}} 2024-04-16T08:07:10.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":50,"type":"Uniter","version":19,"request":"State","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.687Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":50,"response":{"results":[{"uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:10.687Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":51,"type":"Uniter","version":19,"request":"State","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":51,"response":{"results":[{"uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":52,"type":"SecretsManager","version":2,"request":"GetSecretMetadata","params":{}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":52,"response":{"results":null}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":53,"type":"Uniter","version":19,"request":"CurrentModel","params":{}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":53,"response":{"name":"test","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","type":"caas"}} 2024-04-16T08:07:10.688Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":54,"type":"Uniter","version":19,"request":"GetPrincipal","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.689Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":54,"response":{"results":[{"result":"","ok":false}]}} 2024-04-16T08:07:10.689Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":55,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.689Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":55,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:10.689Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":56,"type":"Uniter","version":19,"request":"CharmURL","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":56,"response":{"results":[{"result":"ch:amd64/focal/kubeflow-volumes-289","ok":false}]}} 2024-04-16T08:07:10.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":57,"type":"Uniter","version":19,"request":"State","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":57,"response":{"results":[{"uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 uniter.go:932 starting local juju-exec listener on {unix /var/lib/juju/agents/unit-kubeflow-volumes-2/run.socket } 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.juju.sockets sockets.go:65 ignoring error on removing "/var/lib/juju/agents/unit-kubeflow-volumes-2/run.socket": remove /var/lib/juju/agents/unit-kubeflow-volumes-2/run.socket: no such file or directory 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2 uniter.go:363 unit "kubeflow-volumes/2" started 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":58,"type":"Uniter","version":19,"request":"CanApplyLXDProfile","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 runlistener.go:117 juju-exec listener running 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":58,"response":{"results":[{"result":false}]}} 2024-04-16T08:07:10.691Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":59,"type":"Uniter","version":19,"request":"CharmURL","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.692Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":59,"response":{"results":[{"result":"ch:amd64/focal/kubeflow-volumes-289","ok":true}]}} 2024-04-16T08:07:10.692Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":60,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.692Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":60,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:10.692Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":61,"type":"Uniter","version":19,"request":"CharmModifiedVersion","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":61,"response":{"results":[{"result":0}]}} 2024-04-16T08:07:10.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2 uniter.go:389 hooks are retried true 2024-04-16T08:07:10.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":62,"type":"Uniter","version":19,"request":"Refresh","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.694Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":62,"response":{"Results":[{"Life":"alive","Resolved":"","Error":null}]}} 2024-04-16T08:07:10.694Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":63,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.694Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":63,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:10.694Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:380 starting remote state watcher, actions for unit-kubeflow-volumes-2; blocked=true 2024-04-16T08:07:10.694Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":64,"type":"Uniter","version":19,"request":"Watch","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.705Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":64,"response":{"results":[{"NotifyWatcherId":"12"}]}} 2024-04-16T08:07:10.705Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":65,"type":"Uniter","version":19,"request":"WatchConfigSettingsHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.705Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":66,"type":"NotifyWatcher","version":1,"id":"12","request":"Next","params":{}} 2024-04-16T08:07:10.706Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":65,"response":{"results":[{"watcher-id":"13","changes":["fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015"]}]}} 2024-04-16T08:07:10.706Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":67,"type":"Uniter","version":19,"request":"WatchTrustConfigSettingsHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.706Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":68,"type":"StringsWatcher","version":1,"id":"13","request":"Next","params":{}} 2024-04-16T08:07:10.707Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":67,"response":{"results":[{"watcher-id":"14","changes":["460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e"]}]}} 2024-04-16T08:07:10.707Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":69,"type":"Uniter","version":19,"request":"WatchUnitRelations","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.707Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":70,"type":"StringsWatcher","version":1,"id":"14","request":"Next","params":{}} 2024-04-16T08:07:10.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":69,"response":{"results":[{"watcher-id":"15"}]}} 2024-04-16T08:07:10.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":71,"type":"Uniter","version":19,"request":"WatchUnitAddressesHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":72,"type":"StringsWatcher","version":1,"id":"15","request":"Next","params":{}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":71,"response":{"results":[{"watcher-id":"16","changes":[""]}]}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":73,"type":"SecretsManager","version":2,"request":"WatchConsumedSecretsChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":74,"type":"StringsWatcher","version":1,"id":"16","request":"Next","params":{}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":73,"response":{"results":[{"watcher-id":"17"}]}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":75,"type":"Uniter","version":19,"request":"Watch","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":76,"type":"StringsWatcher","version":1,"id":"17","request":"Next","params":{}} 2024-04-16T08:07:10.721Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":75,"response":{"results":[{"NotifyWatcherId":"18"}]}} 2024-04-16T08:07:10.721Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":77,"type":"Uniter","version":19,"request":"WatchUnitStorageAttachments","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.721Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":78,"type":"NotifyWatcher","version":1,"id":"18","request":"Next","params":{}} 2024-04-16T08:07:10.723Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":77,"response":{"results":[{"watcher-id":"19"}]}} 2024-04-16T08:07:10.723Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":79,"type":"Uniter","version":19,"request":"WatchLeadershipSettings","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.723Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":80,"type":"StringsWatcher","version":1,"id":"19","request":"Next","params":{}} 2024-04-16T08:07:10.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":79,"response":{"results":[{"NotifyWatcherId":"20"}]}} 2024-04-16T08:07:10.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":81,"type":"Uniter","version":19,"request":"WatchActionNotifications","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":82,"type":"NotifyWatcher","version":1,"id":"20","request":"Next","params":{}} 2024-04-16T08:07:10.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":81,"response":{"results":[{"watcher-id":"21"}]}} 2024-04-16T08:07:10.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":83,"type":"Uniter","version":19,"request":"WatchForModelConfigChanges","params":{}} 2024-04-16T08:07:10.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":84,"type":"StringsWatcher","version":1,"id":"21","request":"Next","params":{}} 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":83,"response":{"NotifyWatcherId":"22"}} 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":85,"type":"NotifyWatcher","version":1,"id":"22","request":"Next","params":{}} 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1088 starting secrets rotation watcher 2024-04-16T08:07:10.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1101 starting secret revisions expiry watcher 2024-04-16T08:07:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":86,"type":"SecretsManager","version":2,"request":"WatchSecretsRotationChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1114 starting obsolete secret revisions watcher 2024-04-16T08:07:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":87,"type":"SecretsManager","version":2,"request":"WatchSecretRevisionsExpiryChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":88,"type":"SecretsManager","version":2,"request":"WatchObsolete","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":87,"response":{"watcher-id":"23","changes":[]}} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":88,"response":{"watcher-id":"24"}} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.secretsexpire secretexpire.go:184 got revision expiry secret changes: []watcher.SecretTriggerChange{} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":86,"response":{"watcher-id":"25","changes":[]}} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":89,"type":"SecretsTriggerWatcher","version":1,"id":"23","request":"Next","params":{}} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:615 got secrets change for kubeflow-volumes/2: [] 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.secretsrotate secretrotate.go:169 got rotate secret changes: []watcher.SecretTriggerChange{} 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:166 kubeflow-volumes/2 got wait request for kubeflow-volumes leadership loss 2024-04-16T08:07:10.749Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.leadership tracker.go:322 waiting for kubeflow-volumes/2 to lose kubeflow-volumes leadership 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":90,"type":"SecretsTriggerWatcher","version":1,"id":"25","request":"Next","params":{}} 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":91,"type":"StringsWatcher","version":1,"id":"24","request":"Next","params":{}} 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":92,"type":"SecretsManager","version":2,"request":"GetConsumerSecretsRevisionInfo","params":{"consumer-tag":"unit-kubeflow-volumes-2","uris":null}} 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":92,"response":{"results":[]}} 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:982 got latest secret info: map[string]secrets.SecretRevisionInfo{} 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:993 deleted secrets: [] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:994 obsolete secrets: map[] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:690 got address change for kubeflow-volumes/2: ok=true, hashes=[] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:790 got obsolete secret revisions change for kubeflow-volumes: [] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1018 obsolete secret revisions: map[] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1019 deleted secrets: [] 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:605 got application change for kubeflow-volumes/2 2024-04-16T08:07:10.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":93,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.752Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":93,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:10.752Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":94,"type":"Uniter","version":19,"request":"CharmURL","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.753Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":94,"response":{"results":[{"result":"ch:amd64/focal/kubeflow-volumes-289","ok":false}]}} 2024-04-16T08:07:10.753Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":95,"type":"Uniter","version":19,"request":"CharmModifiedVersion","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":95,"response":{"results":[{"result":0}]}} 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:669 got trust config change for kubeflow-volumes/2: ok=true, hashes=[460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e] 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:701 got leader settings change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:658 got config change for kubeflow-volumes/2: ok=true, hashes=[fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015] 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:739 got update status interval change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:10.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":96,"type":"Uniter","version":19,"request":"ModelConfig","params":{}} 2024-04-16T08:07:10.755Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":96,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:10.756Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:719 got relations change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:10.756Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:729 got storage change for kubeflow-volumes/2: [] ok=true 2024-04-16T08:07:10.756Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":97,"type":"Uniter","version":19,"request":"StorageAttachmentLife","params":{"ids":[]}} 2024-04-16T08:07:10.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":97,"response":{"results":[]}} 2024-04-16T08:07:10.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:595 got unit change for kubeflow-volumes/2 2024-04-16T08:07:10.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":98,"type":"Uniter","version":19,"request":"Refresh","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":98,"response":{"Results":[{"Life":"alive","Resolved":"","Error":null}]}} 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:711 got action change for kubeflow-volumes/2: [] ok=true 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:242 charmdir: available=true opState: started=true stopped=false changing=false 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:304 start to re-download charm ch:amd64/focal/kubeflow-volumes-289 because charm dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" has gone which is usually caused by operator pod re-scheduling 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:85 running operation upgrade to ch:amd64/focal/kubeflow-volumes-289 for kubeflow-volumes/2 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:96 no machine lock needed for kubeflow-volumes/2 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 preparing operation "upgrade to ch:amd64/focal/kubeflow-volumes-289" for kubeflow-volumes/2 2024-04-16T08:07:10.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":99,"type":"Uniter","version":19,"request":"CharmArchiveSha256","params":{"urls":[{"url":"ch:amd64/focal/kubeflow-volumes-289"}]}} 2024-04-16T08:07:10.761Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":99,"response":{"results":[{"result":"e7404046f5d59a0cc8fb2de6a7f1d1fabb016466bab76df228605516e33ef6a7"}]}} 2024-04-16T08:07:10.761Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.charm bundles.go:82 downloading ch:amd64/focal/kubeflow-volumes-289 from API server 2024-04-16T08:07:10.761Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.downloader download.go:112 downloading from ch:amd64/focal/kubeflow-volumes-289 2024-04-16T08:07:10.806Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.downloader download.go:95 download complete ("ch:amd64/focal/kubeflow-volumes-289") 2024-04-16T08:07:10.826Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 INFO juju.downloader download.go:175 download verified ("ch:amd64/focal/kubeflow-volumes-289") 2024-04-16T08:07:10.835Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":100,"type":"Uniter","version":19,"request":"SetCharmURL","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","charm-url":"ch:amd64/focal/kubeflow-volumes-289"}]}} 2024-04-16T08:07:10.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":100,"response":{"results":[{}]}} 2024-04-16T08:07:10.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":101,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: upgrade\nopstep: pending\ncharm: ch:amd64/focal/kubeflow-volumes-289\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:10.840Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":101,"response":{"results":[{}]}} 2024-04-16T08:07:10.840Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 executing operation "upgrade to ch:amd64/focal/kubeflow-volumes-289" for kubeflow-volumes/2 2024-04-16T08:07:10.840Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.charm manifest_deployer.go:143 preparing to deploy charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:10.842Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:10 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.charm manifest_deployer.go:132 deploying charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:12.468Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:12 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:12.471Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:12 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:12.473Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:12 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:13.055Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:13 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:13.076Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:13 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:13.089Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:13 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:13.454Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:13 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:14.054Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:14 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:14.265Z [jujud] 2024-04-16 08:07:14 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:14.265Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:14 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:14.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.charm manifest_deployer.go:164 finishing deploy of charm "ch:amd64/focal/kubeflow-volumes-289" 2024-04-16T08:07:14.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":102,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: upgrade\nopstep: done\ncharm: ch:amd64/focal/kubeflow-volumes-289\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:14.760Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":102,"response":{"results":[{}]}} 2024-04-16T08:07:14.760Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 committing operation "upgrade to ch:amd64/focal/kubeflow-volumes-289" for kubeflow-volumes/2 2024-04-16T08:07:14.761Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":103,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: queued\nhook:\n kind: upgrade-charm\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":103,"response":{"results":[{}]}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":104,"type":"NotifyWatcher","version":1,"id":"12","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":105,"type":"StringsWatcher","version":1,"id":"19","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":106,"type":"NotifyWatcher","version":1,"id":"20","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":107,"type":"NotifyWatcher","version":1,"id":"22","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":108,"type":"StringsWatcher","version":1,"id":"21","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":109,"type":"StringsWatcher","version":1,"id":"24","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":110,"type":"SecretsTriggerWatcher","version":1,"id":"23","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":111,"type":"NotifyWatcher","version":1,"id":"18","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":112,"type":"SecretsTriggerWatcher","version":1,"id":"25","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":113,"type":"StringsWatcher","version":1,"id":"17","request":"Stop","params":{}} 2024-04-16T08:07:14.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":114,"type":"StringsWatcher","version":1,"id":"14","request":"Stop","params":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":115,"type":"StringsWatcher","version":1,"id":"13","request":"Stop","params":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":116,"type":"StringsWatcher","version":1,"id":"15","request":"Stop","params":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":117,"type":"StringsWatcher","version":1,"id":"16","request":"Stop","params":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":104,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":80,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":66,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":105,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":82,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":106,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":85,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":108,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":107,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":84,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":91,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":78,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":109,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":89,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":74,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":110,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":70,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":116,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":111,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":90,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":112,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":76,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":117,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":113,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":114,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":115,"response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":68,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":72,"error":"watcher has been stopped","error-code":"stopped","response":{}} 2024-04-16T08:07:14.764Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":118,"type":"Uniter","version":19,"request":"Refresh","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.765Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":118,"response":{"Results":[{"Life":"alive","Resolved":"","Error":null}]}} 2024-04-16T08:07:14.765Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":119,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.765Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":119,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:14.765Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:380 starting remote state watcher, actions for unit-kubeflow-volumes-2; blocked=true 2024-04-16T08:07:14.765Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":120,"type":"Uniter","version":19,"request":"Watch","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.776Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":120,"response":{"results":[{"NotifyWatcherId":"26"}]}} 2024-04-16T08:07:14.776Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":121,"type":"Uniter","version":19,"request":"WatchConfigSettingsHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.776Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":122,"type":"NotifyWatcher","version":1,"id":"26","request":"Next","params":{}} 2024-04-16T08:07:14.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":121,"response":{"results":[{"watcher-id":"27","changes":["fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015"]}]}} 2024-04-16T08:07:14.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":123,"type":"Uniter","version":19,"request":"WatchTrustConfigSettingsHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":124,"type":"StringsWatcher","version":1,"id":"27","request":"Next","params":{}} 2024-04-16T08:07:14.778Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":123,"response":{"results":[{"watcher-id":"28","changes":["460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e"]}]}} 2024-04-16T08:07:14.778Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":125,"type":"Uniter","version":19,"request":"WatchUnitRelations","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.778Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":126,"type":"StringsWatcher","version":1,"id":"28","request":"Next","params":{}} 2024-04-16T08:07:14.779Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":125,"response":{"results":[{"watcher-id":"29"}]}} 2024-04-16T08:07:14.779Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":127,"type":"Uniter","version":19,"request":"WatchUnitAddressesHash","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.779Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":128,"type":"StringsWatcher","version":1,"id":"29","request":"Next","params":{}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":127,"response":{"results":[{"watcher-id":"30","changes":[""]}]}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":129,"type":"SecretsManager","version":2,"request":"WatchConsumedSecretsChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":130,"type":"StringsWatcher","version":1,"id":"30","request":"Next","params":{}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":129,"response":{"results":[{"watcher-id":"31"}]}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":131,"type":"Uniter","version":19,"request":"Watch","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.780Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":132,"type":"StringsWatcher","version":1,"id":"31","request":"Next","params":{}} 2024-04-16T08:07:14.791Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":131,"response":{"results":[{"NotifyWatcherId":"32"}]}} 2024-04-16T08:07:14.791Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":133,"type":"Uniter","version":19,"request":"WatchUnitStorageAttachments","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.791Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":134,"type":"NotifyWatcher","version":1,"id":"32","request":"Next","params":{}} 2024-04-16T08:07:14.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":133,"response":{"results":[{"watcher-id":"33"}]}} 2024-04-16T08:07:14.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":135,"type":"Uniter","version":19,"request":"WatchLeadershipSettings","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":136,"type":"StringsWatcher","version":1,"id":"33","request":"Next","params":{}} 2024-04-16T08:07:14.802Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":135,"response":{"results":[{"NotifyWatcherId":"34"}]}} 2024-04-16T08:07:14.802Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":137,"type":"Uniter","version":19,"request":"WatchActionNotifications","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.803Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":138,"type":"NotifyWatcher","version":1,"id":"34","request":"Next","params":{}} 2024-04-16T08:07:14.805Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":137,"response":{"results":[{"watcher-id":"35"}]}} 2024-04-16T08:07:14.805Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":139,"type":"Uniter","version":19,"request":"WatchForModelConfigChanges","params":{}} 2024-04-16T08:07:14.805Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":140,"type":"StringsWatcher","version":1,"id":"35","request":"Next","params":{}} 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":139,"response":{"NotifyWatcherId":"36"}} 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":141,"type":"NotifyWatcher","version":1,"id":"36","request":"Next","params":{}} 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1088 starting secrets rotation watcher 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1101 starting secret revisions expiry watcher 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":142,"type":"SecretsManager","version":2,"request":"WatchSecretsRotationChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1114 starting obsolete secret revisions watcher 2024-04-16T08:07:14.816Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":143,"type":"SecretsManager","version":2,"request":"WatchSecretRevisionsExpiryChanges","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.817Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":144,"type":"SecretsManager","version":2,"request":"WatchObsolete","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"},{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":143,"response":{"watcher-id":"37","changes":[]}} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":142,"response":{"watcher-id":"38","changes":[]}} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":144,"response":{"watcher-id":"39"}} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.secretsrotate secretrotate.go:169 got rotate secret changes: []watcher.SecretTriggerChange{} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:729 got storage change for kubeflow-volumes/2: [] ok=true 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":145,"type":"SecretsTriggerWatcher","version":1,"id":"38","request":"Next","params":{}} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:166 kubeflow-volumes/2 got wait request for kubeflow-volumes leadership loss 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.leadership tracker.go:322 waiting for kubeflow-volumes/2 to lose kubeflow-volumes leadership 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.secretsexpire secretexpire.go:184 got revision expiry secret changes: []watcher.SecretTriggerChange{} 2024-04-16T08:07:14.818Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":146,"type":"StringsWatcher","version":1,"id":"39","request":"Next","params":{}} 2024-04-16T08:07:14.819Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":147,"type":"Uniter","version":19,"request":"StorageAttachmentLife","params":{"ids":[]}} 2024-04-16T08:07:14.819Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":148,"type":"SecretsTriggerWatcher","version":1,"id":"37","request":"Next","params":{}} 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":147,"response":{"results":[]}} 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:701 got leader settings change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:719 got relations change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:658 got config change for kubeflow-volumes/2: ok=true, hashes=[fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015] 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:669 got trust config change for kubeflow-volumes/2: ok=true, hashes=[460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e] 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:595 got unit change for kubeflow-volumes/2 2024-04-16T08:07:14.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":149,"type":"Uniter","version":19,"request":"Refresh","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.821Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":149,"response":{"Results":[{"Life":"alive","Resolved":"","Error":null}]}} 2024-04-16T08:07:14.821Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:711 got action change for kubeflow-volumes/2: [] ok=true 2024-04-16T08:07:14.822Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:605 got application change for kubeflow-volumes/2 2024-04-16T08:07:14.822Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":150,"type":"Uniter","version":19,"request":"Life","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.823Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":150,"response":{"results":[{"life":"alive"}]}} 2024-04-16T08:07:14.823Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":151,"type":"Uniter","version":19,"request":"CharmURL","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.824Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":151,"response":{"results":[{"result":"ch:amd64/focal/kubeflow-volumes-289","ok":false}]}} 2024-04-16T08:07:14.824Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":152,"type":"Uniter","version":19,"request":"CharmModifiedVersion","params":{"entities":[{"tag":"application-kubeflow-volumes"}]}} 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":152,"response":{"results":[{"result":0}]}} 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:790 got obsolete secret revisions change for kubeflow-volumes: [] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1018 obsolete secret revisions: map[] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:1019 deleted secrets: [] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:615 got secrets change for kubeflow-volumes/2: [] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":153,"type":"SecretsManager","version":2,"request":"GetConsumerSecretsRevisionInfo","params":{"consumer-tag":"unit-kubeflow-volumes-2","uris":null}} 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":153,"response":{"results":[]}} 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:982 got latest secret info: map[string]secrets.SecretRevisionInfo{} 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:993 deleted secrets: [] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:994 obsolete secrets: map[] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:690 got address change for kubeflow-volumes/2: ok=true, hashes=[] 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:739 got update status interval change for kubeflow-volumes/2: ok=true 2024-04-16T08:07:14.825Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":154,"type":"Uniter","version":19,"request":"ModelConfig","params":{}} 2024-04-16T08:07:14.826Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":154,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:14.826Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:242 charmdir: available=false opState: started=true stopped=false changing=true 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.upgradeseries resolver.go:48 localState.Kind="run-hook", localState.UpgradeMachineStatus="not started", remoteState.UpgradeMachineStatus="not started" 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:344 create relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:14.827Z [jujud] } 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:347 no create relation operation to run 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:49 checking leadership status 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:84 leadership status is up-to-date 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:61 actions are blocked=true; outdated remote charm=true - have pending actions: [] 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:63 actions are blocked, no in flight actions 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2 resolver.go:185 found queued "upgrade-charm" hook 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:132 running op: run upgrade-charm hook 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:85 running operation run upgrade-charm hook for kubeflow-volumes/2 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:88 acquiring machine lock for kubeflow-volumes/2 2024-04-16T08:07:14.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.machinelock machinelock.go:166 acquire machine lock "machine-lock" for kubeflow-volumes/2 uniter (run upgrade-charm hook) 2024-04-16T08:07:14.828Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.machinelock machinelock.go:176 machine lock "machine-lock" acquired for kubeflow-volumes/2 uniter (run upgrade-charm hook) 2024-04-16T08:07:14.828Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 preparing operation "run upgrade-charm hook" for kubeflow-volumes/2 2024-04-16T08:07:14.828Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":155,"type":"PayloadsHookContext","version":1,"request":"List","params":{"entities":null}} 2024-04-16T08:07:14.829Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":155,"response":{"results":null}} 2024-04-16T08:07:14.829Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":156,"type":"Uniter","version":19,"request":"APIAddresses","params":{}} 2024-04-16T08:07:14.830Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":156,"response":{"result":["10.152.183.39:17070","controller-service.controller-uk8s.svc.cluster.local:17070"]}} 2024-04-16T08:07:14.830Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":157,"type":"Uniter","version":19,"request":"SLALevel","params":{}} 2024-04-16T08:07:14.830Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":157,"response":{"result":"unsupported"}} 2024-04-16T08:07:14.830Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":158,"type":"Uniter","version":19,"request":"CloudAPIVersion","params":{}} 2024-04-16T08:07:14.834Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":158,"response":{"result":"1.25.0"}} 2024-04-16T08:07:14.834Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":159,"type":"Uniter","version":19,"request":"ModelConfig","params":{}} 2024-04-16T08:07:14.835Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":159,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:14.835Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":160,"type":"Uniter","version":19,"request":"GetMeterStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:14.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":160,"response":{"results":[{"code":"AMBER","info":"not set"}]}} 2024-04-16T08:07:14.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":161,"type":"Uniter","version":19,"request":"OpenedPortRangesByEndpoint","params":{}} 2024-04-16T08:07:14.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":161,"response":{"results":[{"error":{"message":"getting opened port ranges for \"application\" not supported","code":"not supported"},"unit-port-ranges":null}]}} 2024-04-16T08:07:14.836Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":162,"type":"SecretsManager","version":2,"request":"GetSecretMetadata","params":{}} 2024-04-16T08:07:14.837Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":162,"response":{"results":null}} 2024-04-16T08:07:14.837Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":163,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: pending\nhook:\n kind: upgrade-charm\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:14.839Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":163,"response":{"results":[{}]}} 2024-04-16T08:07:14.839Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 executing operation "run upgrade-charm hook" for kubeflow-volumes/2 2024-04-16T08:07:14.839Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 agent.go:22 [AGENT-STATUS] executing: running upgrade-charm hook 2024-04-16T08:07:14.839Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":164,"type":"Uniter","version":19,"request":"SetAgentStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"executing","info":"running upgrade-charm hook","data":null}]}} 2024-04-16T08:07:14.843Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":164,"response":{"results":[{}]}} 2024-04-16T08:07:14.844Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 DEBUG juju.worker.uniter.runner runner.go:719 starting jujuc server {unix @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket } 2024-04-16T08:07:14.844Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:14 TRACE juju.juju.sockets sockets.go:65 ignoring error on removing "@/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket": remove @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket: no such file or directory 2024-04-16T08:07:15.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "ops 2.12.0 up and running."] 2024-04-16T08:07:15.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 ops 2.12.0 up and running. 2024-04-16T08:07:15.277Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "INFO" "--" "Running legacy hooks/upgrade-charm."] 2024-04-16T08:07:15.277Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.277Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.277Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 INFO unit.kubeflow-volumes/2.juju-log server.go:325 Running legacy hooks/upgrade-charm. 2024-04-16T08:07:15.442Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "ops 2.12.0 up and running."] 2024-04-16T08:07:15.442Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.442Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.442Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 ops 2.12.0 up and running. 2024-04-16T08:07:15.466Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Charm called itself via hooks/upgrade-charm."] 2024-04-16T08:07:15.466Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.466Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.466Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Charm called itself via hooks/upgrade-charm. 2024-04-16T08:07:15.514Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Legacy hooks/upgrade-charm exited with status 0."] 2024-04-16T08:07:15.514Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.514Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.514Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Legacy hooks/upgrade-charm exited with status 0. 2024-04-16T08:07:15.536Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Using controller storage: JUJU_VERSION=3.4.2"] 2024-04-16T08:07:15.536Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.536Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.536Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Using controller storage: JUJU_VERSION=3.4.2 2024-04-16T08:07:15.559Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Initializing SQLite local storage: /var/lib/juju/agents/unit-kubeflow-volumes-2/charm/.unit-state.db."] 2024-04-16T08:07:15.559Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.559Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.559Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Initializing SQLite local storage: /var/lib/juju/agents/unit-kubeflow-volumes-2/charm/.unit-state.db. 2024-04-16T08:07:15.586Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Emitting Juju event upgrade_charm."] 2024-04-16T08:07:15.586Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.586Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.586Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Emitting Juju event upgrade_charm. 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "is-leader" ["--format=json"] 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "is-leader" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:15.615Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:15.644Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "relation-ids" ["ingress" "--format=json"] 2024-04-16T08:07:15.644Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "relation-ids" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.644Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.666Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "resource-get" ["oci-image"] 2024-04-16T08:07:15.666Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "resource-get" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.666Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":165,"type":"ResourcesHookContext","version":1,"request":"GetResourceInfo","params":{"resource-names":["oci-image"]}} 2024-04-16T08:07:15.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":165,"response":{"resources":[{"resource":{"name":"oci-image","type":"oci-image","path":"","description":"Backing OCI image","origin":"store","revision":263,"fingerprint":"iqk+PDiYfGKl8JjCCPFaiBgVa+py3A5mlFOdgFNqeNlzPFcEZ1y9YBl/uPrWhQpb","size":495,"id":"kubeflow-volumes/oci-image","pending-id":"","application":"kubeflow-volumes","username":"kubeflow-volumes/2","timestamp":"2024-04-16T07:06:35.978Z"}}]}} 2024-04-16T08:07:15.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "status-set" ["--application=False" "maintenance" "Setting pod spec"] 2024-04-16T08:07:15.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "status-set" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:609 [WORKLOAD-STATUS] maintenance: Setting pod spec 2024-04-16T08:07:15.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":166,"type":"Uniter","version":19,"request":"SetUnitStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"maintenance","info":"Setting pod spec","data":null}]}} 2024-04-16T08:07:15.713Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":166,"response":{"results":[{}]}} 2024-04-16T08:07:15.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "config-get" ["--format=json"] 2024-04-16T08:07:15.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "config-get" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.734Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":167,"type":"Uniter","version":19,"request":"ConfigSettings","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:15.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":167,"response":{"results":[{"settings":{"backend-mode":"development","port":5000,"secure-cookies":false,"volume-viewer-image":"charmedkubeflow/filebrowser:2.27.0-f21fe9d"}}]}} 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "pod-spec-set" ["--file" "/tmp/tmp_7tkn284-pod-spec-set/spec.yaml" "--k8s-resources" "/tmp/tmp_7tkn284-pod-spec-set/k8s-resources.yaml"] 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "pod-spec-set" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:15.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:15.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG juju.kubernetes.provider.specs decode.go:67 decoding stream as JSON 2024-04-16T08:07:15.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:221 running hook tool "status-set" ["--application=False" "active" ""] 2024-04-16T08:07:15.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 DEBUG jujuc server.go:222 running hook tool "status-set" for kubeflow-volumes/2-upgrade-charm-4653318692433728564 2024-04-16T08:07:15.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-upgrade-charm-4653318692433728564"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:15.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:609 [WORKLOAD-STATUS] active: 2024-04-16T08:07:15.792Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":168,"type":"Uniter","version":19,"request":"SetUnitStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"active","info":"","data":null}]}} 2024-04-16T08:07:15.797Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:15 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":168,"response":{"results":[{}]}} 2024-04-16T08:07:16.027Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:16.027Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:16.027Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:16.027Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:16.027Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":169,"type":"Uniter","version":19,"request":"CommitHookChanges","params":{"args":[{"tag":"unit-kubeflow-volumes-2","update-network-info":false,"pod-spec":{"tag":"application-kubeflow-volumes","spec":"containers:\n- envConfig:\n APP_PREFIX: /volumes\n APP_SECURE_COOKIES: 'false'\n BACKEND_MODE: development\n USERID_HEADER: kubeflow-userid\n USERID_PREFIX: ''\n VOLUME_VIEWER_IMAGE: charmedkubeflow/filebrowser:2.27.0-f21fe9d\n imageDetails:\n imagePath: registry.jujucharms.com/charm/6tvhkd5az86qo4m4zxeoqip1iejoxusiphggc/oci-image@sha256:2261827b4899292d193310a85b529e14d58436beda4f7cfd3b0903494c70f0d9\n password: MDAxOGxvY2F0aW9uIGNoYXJtc3RvcmUKMDAzMGlkZW50aWZpZXIgOTBlODUwYWU2MzllNTZmMGU2OTc5MmYyOTQzZWY4ZDYKMDA0ZGNpZCBpcy1kb2NrZXItcmVwbyBjaGFybS82dHZoa2Q1YXo4NnFvNG00enhlb3FpcDFpZWpveHVzaXBoZ2djL29jaS1pbWFnZQowMDEzY2lkIGFsbG93IHB1bGwKMDAyZnNpZ25hdHVyZSCvOL2oxWC33wjxwX4MY8ldgrLgMOF+WHxWVcYZ6CF02Qo\n username: docker-registry\n name: volumes-web-app\n ports:\n - containerPort: 5000\n name: http\n volumeConfig:\n - files:\n - content: \"# Source: manifests/apps/volumes-web-app/upstream/base/viewer-spec.yaml\\n#\n Note: the volumes-web-app allows expanding strings using ${VAR_NAME}\\n# You\n may use any environment variable. This lets us e.g. specify images that can\n be modified using kustomize's image transformer.\\n# Additionally, 'PVC_NAME',\n 'NAME' and 'NAMESPACE' are defined\\n# Name of the pvc is set by the volumes\n web app\\npvc: $NAME\\npodSpec:\\n serviceAccountName: default-editor\\n containers:\\n\n \\ - name: main\\n image: $VOLUME_VIEWER_IMAGE\\n env:\\n -\n name: FB_ADDRESS\\n value: \\\"0.0.0.0\\\"\\n - name: FB_PORT\\n\n \\ value: \\\"8080\\\"\\n - name: FB_DATABASE\\n value: /tmp/filebrowser.db\\n\n \\ - name: FB_NOAUTH\\n value: \\\"true\\\"\\n - name: FB_BASEURL\\n\n \\ value: /pvcviewers/$NAMESPACE/$NAME/\\n readinessProbe:\\n tcpSocket:\\n\n \\ port: 8080\\n initialDelaySeconds: 2\\n periodSeconds:\n 10\\n # viewer-volume is provided automatically by the volumes web app\\n\n \\ volumeMounts:\\n - name: viewer-volume\\n mountPath: /srv\\n\n \\ workingDir: /srv\\n securityContext:\\n allowPrivilegeEscalation:\n false\\n capabilities:\\n drop:\\n - ALL\\n volumes:\\n\n \\ - name: viewer-volume\\n persistentVolumeClaim:\\n claimName:\n $NAME\\nnetworking:\\n targetPort: 8080\\n basePrefix: \\\"/pvcviewers\\\"\\n rewrite:\n \\\"/\\\"\\n timeout: 30s\\nrwoScheduling: true\"\n path: viewer-spec.yaml\n mountPath: /etc/config/\n name: viewer-spec\nserviceAccount:\n roles:\n - global: true\n rules:\n - apiGroups:\n - ''\n resources:\n - namespaces\n - pods\n verbs:\n - get\n - list\n - apiGroups:\n - authorization.k8s.io\n resources:\n - subjectaccessreviews\n verbs:\n - create\n - apiGroups:\n - ''\n resources:\n - persistentvolumeclaims\n verbs:\n - create\n - delete\n - get\n - list\n - watch\n - update\n - patch\n - apiGroups:\n - storage.k8s.io\n resources:\n - storageclasses\n verbs:\n - get\n - list\n - watch\n - apiGroups:\n - ''\n resources:\n - events\n verbs:\n - list\n - apiGroups:\n - kubeflow.org\n resources:\n - notebooks\n verbs:\n - list\n - apiGroups:\n - kubeflow.org\n resources:\n - pvcviewers\n verbs:\n - get\n - list\n - create\n - delete\nversion: 3\nconfigMaps:\n volumes-web-app-viewer-spec-ck6bhh4bdm:\n viewer-spec.yaml: \"# Source: manifests/apps/volumes-web-app/upstream/base/viewer-spec.yaml\\n#\n Note: the volumes-web-app allows expanding strings using ${VAR_NAME}\\n# You\n may use any environment variable. This lets us e.g. specify images that can\n be modified using kustomize's image transformer.\\n# Additionally, 'PVC_NAME',\n 'NAME' and 'NAMESPACE' are defined\\n# Name of the pvc is set by the volumes\n web app\\npvc: $NAME\\npodSpec:\\n serviceAccountName: default-editor\\n containers:\\n\n \\ - name: main\\n image: $VOLUME_VIEWER_IMAGE\\n env:\\n - name:\n FB_ADDRESS\\n value: \\\"0.0.0.0\\\"\\n - name: FB_PORT\\n value:\n \\\"8080\\\"\\n - name: FB_DATABASE\\n value: /tmp/filebrowser.db\\n\n \\ - name: FB_NOAUTH\\n value: \\\"true\\\"\\n - name: FB_BASEURL\\n\n \\ value: /pvcviewers/$NAMESPACE/$NAME/\\n readinessProbe:\\n tcpSocket:\\n\n \\ port: 8080\\n initialDelaySeconds: 2\\n periodSeconds:\n 10\\n # viewer-volume is provided automatically by the volumes web app\\n\n \\ volumeMounts:\\n - name: viewer-volume\\n mountPath: /srv\\n\n \\ workingDir: /srv\\n securityContext:\\n allowPrivilegeEscalation:\n false\\n capabilities:\\n drop:\\n - ALL\\n volumes:\\n\n \\ - name: viewer-volume\\n persistentVolumeClaim:\\n claimName:\n $NAME\\nnetworking:\\n targetPort: 8080\\n basePrefix: \\\"/pvcviewers\\\"\\n rewrite:\n \\\"/\\\"\\n timeout: 30s\\nrwoScheduling: true\"\n"}}]}} 2024-04-16T08:07:16.031Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":169,"response":{"results":[{}]}} 2024-04-16T08:07:16.031Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:1490 checking for reboot request 2024-04-16T08:07:16.031Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation runhook.go:186 ran "upgrade-charm" hook (via hook dispatching script: dispatch) 2024-04-16T08:07:16.031Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":170,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: done\nhook:\n kind: upgrade-charm\nhook-step: done\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:16.033Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":170,"response":{"results":[{}]}} 2024-04-16T08:07:16.033Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 committing operation "run upgrade-charm hook" for kubeflow-volumes/2 2024-04-16T08:07:16.033Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":171,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: queued\nhook:\n kind: config-changed\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:16.035Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":171,"response":{"results":[{}]}} 2024-04-16T08:07:16.035Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:206 created rotating log file "/var/log/juju/machine-lock.log" with max size 10 MB and max backups 5 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:190 machine lock "machine-lock" released for kubeflow-volumes/2 uniter (run upgrade-charm hook) 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:124 lock released for kubeflow-volumes/2 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:242 charmdir: available=true opState: started=true stopped=false changing=false 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.upgradeseries resolver.go:48 localState.Kind="run-hook", localState.UpgradeMachineStatus="not started", remoteState.UpgradeMachineStatus="not started" 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:344 create relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:16.036Z [jujud] } 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:347 no create relation operation to run 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:49 checking leadership status 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:84 leadership status is up-to-date 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:61 actions are blocked=true; outdated remote charm=true - have pending actions: [] 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:63 actions are blocked, no in flight actions 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2 resolver.go:185 found queued "config-changed" hook 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:132 running op: run config-changed hook 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:85 running operation run config-changed hook for kubeflow-volumes/2 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:88 acquiring machine lock for kubeflow-volumes/2 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:166 acquire machine lock "machine-lock" for kubeflow-volumes/2 uniter (run config-changed hook) 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:176 machine lock "machine-lock" acquired for kubeflow-volumes/2 uniter (run config-changed hook) 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 preparing operation "run config-changed hook" for kubeflow-volumes/2 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":172,"type":"PayloadsHookContext","version":1,"request":"List","params":{"entities":null}} 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":172,"response":{"results":null}} 2024-04-16T08:07:16.036Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":173,"type":"Uniter","version":19,"request":"APIAddresses","params":{}} 2024-04-16T08:07:16.037Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":173,"response":{"result":["10.152.183.39:17070","controller-service.controller-uk8s.svc.cluster.local:17070"]}} 2024-04-16T08:07:16.037Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":174,"type":"Uniter","version":19,"request":"SLALevel","params":{}} 2024-04-16T08:07:16.037Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":174,"response":{"result":"unsupported"}} 2024-04-16T08:07:16.037Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":175,"type":"Uniter","version":19,"request":"CloudAPIVersion","params":{}} 2024-04-16T08:07:16.041Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":175,"response":{"result":"1.25.0"}} 2024-04-16T08:07:16.041Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":176,"type":"Uniter","version":19,"request":"ModelConfig","params":{}} 2024-04-16T08:07:16.042Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":176,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:07:16.042Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":177,"type":"Uniter","version":19,"request":"GetMeterStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:16.043Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":177,"response":{"results":[{"code":"AMBER","info":"not set"}]}} 2024-04-16T08:07:16.043Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":178,"type":"Uniter","version":19,"request":"OpenedPortRangesByEndpoint","params":{}} 2024-04-16T08:07:16.043Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":178,"response":{"results":[{"error":{"message":"getting opened port ranges for \"application\" not supported","code":"not supported"},"unit-port-ranges":null}]}} 2024-04-16T08:07:16.043Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":179,"type":"SecretsManager","version":2,"request":"GetSecretMetadata","params":{}} 2024-04-16T08:07:16.044Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":179,"response":{"results":null}} 2024-04-16T08:07:16.044Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":180,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: pending\nhook:\n kind: config-changed\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:16.046Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":180,"response":{"results":[{}]}} 2024-04-16T08:07:16.046Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 executing operation "run config-changed hook" for kubeflow-volumes/2 2024-04-16T08:07:16.046Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 agent.go:22 [AGENT-STATUS] executing: running config-changed hook 2024-04-16T08:07:16.046Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":181,"type":"Uniter","version":19,"request":"SetAgentStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"executing","info":"running config-changed hook","data":null}]}} 2024-04-16T08:07:16.051Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":181,"response":{"results":[{}]}} 2024-04-16T08:07:16.051Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.uniter.runner runner.go:719 starting jujuc server {unix @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket } 2024-04-16T08:07:16.051Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.juju.sockets sockets.go:65 ignoring error on removing "@/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket": remove @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket: no such file or directory 2024-04-16T08:07:16.207Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "ops 2.12.0 up and running."] 2024-04-16T08:07:16.207Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.207Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.207Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 ops 2.12.0 up and running. 2024-04-16T08:07:16.233Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Emitting Juju event config_changed."] 2024-04-16T08:07:16.233Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.233Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.233Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Emitting Juju event config_changed. 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "is-leader" ["--format=json"] 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "is-leader" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:16.255Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:16.284Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "relation-ids" ["ingress" "--format=json"] 2024-04-16T08:07:16.284Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "relation-ids" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.284Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.306Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "resource-get" ["oci-image"] 2024-04-16T08:07:16.306Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "resource-get" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.306Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.321Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":182,"type":"ResourcesHookContext","version":1,"request":"GetResourceInfo","params":{"resource-names":["oci-image"]}} 2024-04-16T08:07:16.322Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":182,"response":{"resources":[{"resource":{"name":"oci-image","type":"oci-image","path":"","description":"Backing OCI image","origin":"store","revision":263,"fingerprint":"iqk+PDiYfGKl8JjCCPFaiBgVa+py3A5mlFOdgFNqeNlzPFcEZ1y9YBl/uPrWhQpb","size":495,"id":"kubeflow-volumes/oci-image","pending-id":"","application":"kubeflow-volumes","username":"kubeflow-volumes/2","timestamp":"2024-04-16T07:06:35.978Z"}}]}} 2024-04-16T08:07:16.346Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "status-set" ["--application=False" "maintenance" "Setting pod spec"] 2024-04-16T08:07:16.346Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "status-set" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.346Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.346Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:609 [WORKLOAD-STATUS] maintenance: Setting pod spec 2024-04-16T08:07:16.346Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":183,"type":"Uniter","version":19,"request":"SetUnitStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"maintenance","info":"Setting pod spec","data":null}]}} 2024-04-16T08:07:16.351Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":183,"response":{"results":[{}]}} 2024-04-16T08:07:16.378Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "config-get" ["--format=json"] 2024-04-16T08:07:16.378Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "config-get" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.378Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.378Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":184,"type":"Uniter","version":19,"request":"ConfigSettings","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:07:16.379Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":184,"response":{"results":[{"settings":{"backend-mode":"development","port":5000,"secure-cookies":false,"volume-viewer-image":"charmedkubeflow/filebrowser:2.27.0-f21fe9d"}}]}} 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "pod-spec-set" ["--file" "/tmp/tmphd6t9u_v-pod-spec-set/spec.yaml" "--k8s-resources" "/tmp/tmphd6t9u_v-pod-spec-set/k8s-resources.yaml"] 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "pod-spec-set" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:16.406Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.kubernetes.provider.specs decode.go:67 decoding stream as JSON 2024-04-16T08:07:16.431Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:221 running hook tool "status-set" ["--application=False" "active" ""] 2024-04-16T08:07:16.431Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG jujuc server.go:222 running hook tool "status-set" for kubeflow-volumes/2-config-changed-8326047210252455702 2024-04-16T08:07:16.431Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-config-changed-8326047210252455702"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:07:16.431Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:609 [WORKLOAD-STATUS] active: 2024-04-16T08:07:16.431Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":185,"type":"Uniter","version":19,"request":"SetUnitStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"active","info":"","data":null}]}} 2024-04-16T08:07:16.435Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":185,"response":{"results":[{}]}} 2024-04-16T08:07:16.660Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:07:16.660Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:07:16.660Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:07:16.660Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:07:16.660Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":186,"type":"Uniter","version":19,"request":"CommitHookChanges","params":{"args":[{"tag":"unit-kubeflow-volumes-2","update-network-info":true,"pod-spec":{"tag":"application-kubeflow-volumes","spec":"containers:\n- envConfig:\n APP_PREFIX: /volumes\n APP_SECURE_COOKIES: 'false'\n BACKEND_MODE: development\n USERID_HEADER: kubeflow-userid\n USERID_PREFIX: ''\n VOLUME_VIEWER_IMAGE: charmedkubeflow/filebrowser:2.27.0-f21fe9d\n imageDetails:\n imagePath: registry.jujucharms.com/charm/6tvhkd5az86qo4m4zxeoqip1iejoxusiphggc/oci-image@sha256:2261827b4899292d193310a85b529e14d58436beda4f7cfd3b0903494c70f0d9\n password: MDAxOGxvY2F0aW9uIGNoYXJtc3RvcmUKMDAzMGlkZW50aWZpZXIgOTBlODUwYWU2MzllNTZmMGU2OTc5MmYyOTQzZWY4ZDYKMDA0ZGNpZCBpcy1kb2NrZXItcmVwbyBjaGFybS82dHZoa2Q1YXo4NnFvNG00enhlb3FpcDFpZWpveHVzaXBoZ2djL29jaS1pbWFnZQowMDEzY2lkIGFsbG93IHB1bGwKMDAyZnNpZ25hdHVyZSCvOL2oxWC33wjxwX4MY8ldgrLgMOF+WHxWVcYZ6CF02Qo\n username: docker-registry\n name: volumes-web-app\n ports:\n - containerPort: 5000\n name: http\n volumeConfig:\n - files:\n - content: \"# Source: manifests/apps/volumes-web-app/upstream/base/viewer-spec.yaml\\n#\n Note: the volumes-web-app allows expanding strings using ${VAR_NAME}\\n# You\n may use any environment variable. This lets us e.g. specify images that can\n be modified using kustomize's image transformer.\\n# Additionally, 'PVC_NAME',\n 'NAME' and 'NAMESPACE' are defined\\n# Name of the pvc is set by the volumes\n web app\\npvc: $NAME\\npodSpec:\\n serviceAccountName: default-editor\\n containers:\\n\n \\ - name: main\\n image: $VOLUME_VIEWER_IMAGE\\n env:\\n -\n name: FB_ADDRESS\\n value: \\\"0.0.0.0\\\"\\n - name: FB_PORT\\n\n \\ value: \\\"8080\\\"\\n - name: FB_DATABASE\\n value: /tmp/filebrowser.db\\n\n \\ - name: FB_NOAUTH\\n value: \\\"true\\\"\\n - name: FB_BASEURL\\n\n \\ value: /pvcviewers/$NAMESPACE/$NAME/\\n readinessProbe:\\n tcpSocket:\\n\n \\ port: 8080\\n initialDelaySeconds: 2\\n periodSeconds:\n 10\\n # viewer-volume is provided automatically by the volumes web app\\n\n \\ volumeMounts:\\n - name: viewer-volume\\n mountPath: /srv\\n\n \\ workingDir: /srv\\n securityContext:\\n allowPrivilegeEscalation:\n false\\n capabilities:\\n drop:\\n - ALL\\n volumes:\\n\n \\ - name: viewer-volume\\n persistentVolumeClaim:\\n claimName:\n $NAME\\nnetworking:\\n targetPort: 8080\\n basePrefix: \\\"/pvcviewers\\\"\\n rewrite:\n \\\"/\\\"\\n timeout: 30s\\nrwoScheduling: true\"\n path: viewer-spec.yaml\n mountPath: /etc/config/\n name: viewer-spec\nserviceAccount:\n roles:\n - global: true\n rules:\n - apiGroups:\n - ''\n resources:\n - namespaces\n - pods\n verbs:\n - get\n - list\n - apiGroups:\n - authorization.k8s.io\n resources:\n - subjectaccessreviews\n verbs:\n - create\n - apiGroups:\n - ''\n resources:\n - persistentvolumeclaims\n verbs:\n - create\n - delete\n - get\n - list\n - watch\n - update\n - patch\n - apiGroups:\n - storage.k8s.io\n resources:\n - storageclasses\n verbs:\n - get\n - list\n - watch\n - apiGroups:\n - ''\n resources:\n - events\n verbs:\n - list\n - apiGroups:\n - kubeflow.org\n resources:\n - notebooks\n verbs:\n - list\n - apiGroups:\n - kubeflow.org\n resources:\n - pvcviewers\n verbs:\n - get\n - list\n - create\n - delete\nversion: 3\nconfigMaps:\n volumes-web-app-viewer-spec-ck6bhh4bdm:\n viewer-spec.yaml: \"# Source: manifests/apps/volumes-web-app/upstream/base/viewer-spec.yaml\\n#\n Note: the volumes-web-app allows expanding strings using ${VAR_NAME}\\n# You\n may use any environment variable. This lets us e.g. specify images that can\n be modified using kustomize's image transformer.\\n# Additionally, 'PVC_NAME',\n 'NAME' and 'NAMESPACE' are defined\\n# Name of the pvc is set by the volumes\n web app\\npvc: $NAME\\npodSpec:\\n serviceAccountName: default-editor\\n containers:\\n\n \\ - name: main\\n image: $VOLUME_VIEWER_IMAGE\\n env:\\n - name:\n FB_ADDRESS\\n value: \\\"0.0.0.0\\\"\\n - name: FB_PORT\\n value:\n \\\"8080\\\"\\n - name: FB_DATABASE\\n value: /tmp/filebrowser.db\\n\n \\ - name: FB_NOAUTH\\n value: \\\"true\\\"\\n - name: FB_BASEURL\\n\n \\ value: /pvcviewers/$NAMESPACE/$NAME/\\n readinessProbe:\\n tcpSocket:\\n\n \\ port: 8080\\n initialDelaySeconds: 2\\n periodSeconds:\n 10\\n # viewer-volume is provided automatically by the volumes web app\\n\n \\ volumeMounts:\\n - name: viewer-volume\\n mountPath: /srv\\n\n \\ workingDir: /srv\\n securityContext:\\n allowPrivilegeEscalation:\n false\\n capabilities:\\n drop:\\n - ALL\\n volumes:\\n\n \\ - name: viewer-volume\\n persistentVolumeClaim:\\n claimName:\n $NAME\\nnetworking:\\n targetPort: 8080\\n basePrefix: \\\"/pvcviewers\\\"\\n rewrite:\n \\\"/\\\"\\n timeout: 30s\\nrwoScheduling: true\"\n"}}]}} 2024-04-16T08:07:16.671Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":186,"response":{"results":[{}]}} 2024-04-16T08:07:16.671Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:1490 checking for reboot request 2024-04-16T08:07:16.671Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation runhook.go:186 ran "config-changed" hook (via hook dispatching script: dispatch) 2024-04-16T08:07:16.671Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":187,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: done\nhook:\n kind: config-changed\nhook-step: done\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:16.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":187,"response":{"results":[{}]}} 2024-04-16T08:07:16.678Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 committing operation "run config-changed hook" for kubeflow-volumes/2 2024-04-16T08:07:16.679Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":188,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":188,"response":{"results":[{}]}} 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:206 created rotating log file "/var/log/juju/machine-lock.log" with max size 10 MB and max backups 5 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.machinelock machinelock.go:190 machine lock "machine-lock" released for kubeflow-volumes/2 uniter (run config-changed hook) 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:124 lock released for kubeflow-volumes/2 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:242 charmdir: available=true opState: started=true stopped=false changing=false 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.upgradeseries resolver.go:48 localState.Kind="continue", localState.UpgradeMachineStatus="not started", remoteState.UpgradeMachineStatus="not started" 2024-04-16T08:07:16.685Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:344 create relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:16.685Z [jujud] } 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:16.686Z [jujud] } 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:347 no create relation operation to run 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:49 checking leadership status 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:84 leadership status is up-to-date 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:61 actions are blocked=true; outdated remote charm=true - have pending actions: [] 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:63 actions are blocked, no in flight actions 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 resolver.go:216 no operations in progress; waiting for changes 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:56 relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:16.686Z [jujud] } 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:07:16.686Z [jujud] } 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:59 no relation operation to run 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 agent.go:22 [AGENT-STATUS] idle: 2024-04-16T08:07:16.686Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":189,"type":"Uniter","version":19,"request":"SetAgentStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2","status":"idle","info":"","data":null}]}} 2024-04-16T08:07:16.692Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:16 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":189,"response":{"results":[{}]}} 2024-04-16T08:07:18.475Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:18 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:18.478Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:18 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:18.480Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:18 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:19.054Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:19.065Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:19.085Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:19.088Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:19.453Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:19.854Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:19.863Z [jujud] 2024-04-16 08:07:19 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:19.863Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:19 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:25.345Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:25.348Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:25.350Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:25.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:07:25.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:07:25.690Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":190,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:07:25.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":190,"response":{"results":[{}]}} 2024-04-16T08:07:25.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:07:55.689924508 +0000 UTC m=+59.133239902 2024-04-16T08:07:25.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:25 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:07:40.689924508 +0000 UTC m=+44.133239902 2024-04-16T08:07:25.902Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:25.912Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:25.952Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:25 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:26.301Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:26 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:26.901Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:26 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:27.113Z [jujud] 2024-04-16 08:07:27 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:27.113Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:27 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:27.275Z [jujud] 2024-04-16 08:07:27 INFO juju.state logs.go:664 LogTailer starting oplog tailing: recent id count=4819, lastTime=2024-04-16 08:07:26.304567407 +0000 UTC, minOplogTs=2024-04-16 08:06:26.304567407 +0000 UTC 2024-04-16T08:07:27.275Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:27 INFO juju.state logs.go:664 LogTailer starting oplog tailing: recent id count=4819, lastTime=2024-04-16 08:07:26.304567407 +0000 UTC, minOplogTs=2024-04-16 08:06:26.304567407 +0000 UTC 2024-04-16T08:07:31.015Z [pebble] GET /v1/notices?timeout=30s 30.000647528s 200 2024-04-16T08:07:33.248Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:33.253Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:33.254Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:33.827Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:33.837Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:33.861Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:33 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:34.227Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:34 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:34.628Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:34 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:34.634Z [jujud] 2024-04-16 08:07:34 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:34.634Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:34 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:40.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:07:40.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:07:40.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":191,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:07:40.701Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":191,"response":{"results":[{}]}} 2024-04-16T08:07:40.701Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:08:10.692967282 +0000 UTC m=+74.136282668 2024-04-16T08:07:40.701Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:40 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:07:55.692967282 +0000 UTC m=+59.136282668 2024-04-16T08:07:41.730Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:41 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:41.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:41 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:41.910Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:41 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:42.311Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:42 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:42.322Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:42 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:42.345Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:42 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:42.710Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:42 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:43.110Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:43 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:43.118Z [jujud] 2024-04-16 08:07:43 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:43.119Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:43 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:52.084Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:52.088Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:07:52.089Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:52.662Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:07:52.673Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:07:52.693Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:52 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:07:53.062Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:53 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:53.661Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:07:53 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:07:53.869Z [jujud] 2024-04-16 08:07:53 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:53.869Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:07:53 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:07:55.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:07:55.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:07:55.697Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":192,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:07:55.704Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":192,"response":{"results":[{}]}} 2024-04-16T08:07:55.704Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:08:25.697101272 +0000 UTC m=+89.140416653 2024-04-16T08:07:55.704Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:07:55 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:08:10.697101272 +0000 UTC m=+74.140416653 2024-04-16T08:08:01.016Z [pebble] GET /v1/notices?timeout=30s 30.000956223s 200 2024-04-16T08:08:05.543Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:05 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:05.549Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:05 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:05.551Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:05 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:06.120Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:06.131Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:08:06.152Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:08:06.238Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":18,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:08:06.238Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":18,"response":{}} 2024-04-16T08:08:06.519Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:06 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:06.916Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":193,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:08:06.916Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":193,"response":{}} 2024-04-16T08:08:07.120Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:07 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:07.329Z [jujud] 2024-04-16 08:08:07 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:07.329Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:08:07 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:10.699Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:08:10.699Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:08:10.699Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":194,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:08:10.708Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":194,"response":{"results":[{}]}} 2024-04-16T08:08:10.708Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:08:40.699540701 +0000 UTC m=+104.142856082 2024-04-16T08:08:10.708Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:08:25.699540701 +0000 UTC m=+89.142856082 2024-04-16T08:08:19.398Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:19.401Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:19.583Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:19.986Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:19.997Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:08:20.016Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:08:20.385Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:20.786Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:20.795Z [jujud] 2024-04-16 08:08:20 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:20.795Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:08:20 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:25.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:08:25.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:08:25.709Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":195,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:08:25.717Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":195,"response":{"results":[{}]}} 2024-04-16T08:08:25.717Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:08:55.708747913 +0000 UTC m=+119.152063316 2024-04-16T08:08:25.717Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:25 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:08:40.708747913 +0000 UTC m=+104.152063316 2024-04-16T08:08:31.018Z [pebble] GET /v1/notices?timeout=30s 30.000756945s 200 2024-04-16T08:08:35.894Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:35 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:35.916Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:35 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:36.073Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:36 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:36.473Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:36 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:36.484Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:36 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:08:36.507Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:36 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:08:36.873Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:36 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:37.273Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:37 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:37.282Z [jujud] 2024-04-16 08:08:37 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:37.282Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:08:37 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:40.713Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:08:40.713Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:08:40.713Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":196,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:08:40.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":196,"response":{"results":[{}]}} 2024-04-16T08:08:40.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:09:10.712585528 +0000 UTC m=+134.155900927 2024-04-16T08:08:40.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:40 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:08:55.712585528 +0000 UTC m=+119.155900927 2024-04-16T08:08:55.664Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:55.667Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:08:55.669Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:55.718Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:08:55.718Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:08:55.718Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":197,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:08:55.722Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":197,"response":{"results":[{}]}} 2024-04-16T08:08:55.722Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:09:25.718673714 +0000 UTC m=+149.161989058 2024-04-16T08:08:55.722Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:08:55 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:09:10.718673714 +0000 UTC m=+134.161989058 2024-04-16T08:08:55.820Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:08:55.824Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:08:55.834Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:55 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:08:56.221Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:56 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:57.021Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:08:57 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:08:57.030Z [jujud] 2024-04-16 08:08:57 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:08:57.030Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:08:57 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:09:01.019Z [pebble] GET /v1/notices?timeout=30s 30.000450938s 200 2024-04-16T08:09:06.239Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":19,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:09:06.239Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":19,"response":{}} 2024-04-16T08:09:06.917Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":198,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:09:06.917Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":198,"response":{}} 2024-04-16T08:09:10.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:09:10.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:09:10.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":199,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:09:10.744Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":199,"response":{"results":[{}]}} 2024-04-16T08:09:10.744Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:09:40.719852795 +0000 UTC m=+164.163168176 2024-04-16T08:09:10.744Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:09:25.719852795 +0000 UTC m=+149.163168176 2024-04-16T08:09:19.823Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:09:19.829Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:09:19.831Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:19 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:09:20.402Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:09:20.412Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:09:20.434Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:09:20.801Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:09:21.402Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:21 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:09:21.611Z [jujud] 2024-04-16 08:09:21 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:09:21.612Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:09:21 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:09:25.727Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:09:25.727Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:09:25.727Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":200,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:09:25.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":200,"response":{"results":[{}]}} 2024-04-16T08:09:25.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:09:55.726866392 +0000 UTC m=+179.170181775 2024-04-16T08:09:25.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:25 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:09:40.726866392 +0000 UTC m=+164.170181775 2024-04-16T08:09:31.020Z [pebble] GET /v1/notices?timeout=30s 30.000995009s 200 2024-04-16T08:09:40.728Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:09:40.728Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:09:40.728Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":201,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:09:40.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":201,"response":{"results":[{}]}} 2024-04-16T08:09:40.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:10:10.727652628 +0000 UTC m=+194.170968025 2024-04-16T08:09:40.735Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:40 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:09:55.727652628 +0000 UTC m=+179.170968025 2024-04-16T08:09:42.728Z [jujud] 2024-04-16 08:09:42 ERROR juju.apiserver debuglog.go:126 debug-log handler error: sending failed: write tcp 127.0.0.1:17070->127.0.0.1:41580: write: broken pipe 2024-04-16T08:09:42.729Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:09:42 ERROR juju.apiserver debuglog.go:126 debug-log handler error: sending failed: write tcp 127.0.0.1:17070->127.0.0.1:41580: write: broken pipe 2024-04-16T08:09:48.738Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:09:48.743Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:09:48.745Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:09:49.313Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:49 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:09:49.319Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:49 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:09:49.333Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:49 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:09:49.712Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:49 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:09:50.312Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:09:50 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:09:50.521Z [jujud] 2024-04-16 08:09:50 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:09:50.521Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:09:50 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:09:55.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:09:55.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:09:55.736Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":202,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:09:55.740Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":202,"response":{"results":[{}]}} 2024-04-16T08:09:55.740Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:10:25.736644312 +0000 UTC m=+209.179959667 2024-04-16T08:09:55.740Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:09:55 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:10:10.736644312 +0000 UTC m=+194.179959667 2024-04-16T08:10:01.021Z [pebble] GET /v1/notices?timeout=30s 30.000445115s 200 2024-04-16T08:10:06.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":20,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:10:06.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":20,"response":{}} 2024-04-16T08:10:06.918Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":203,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:10:06.919Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":203,"response":{}} 2024-04-16T08:10:10.739Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:10:10.739Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:10:10.739Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":204,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:10:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":204,"response":{"results":[{}]}} 2024-04-16T08:10:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:10:40.738879621 +0000 UTC m=+224.182195027 2024-04-16T08:10:10.748Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:10:25.738879621 +0000 UTC m=+209.182195027 2024-04-16T08:10:20.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:10:20.754Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:10:20.756Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:20 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:10:21.332Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:21 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:10:21.343Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:21 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:10:21.367Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:21 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:10:21.730Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:21 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:10:22.330Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:22 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:10:22.542Z [jujud] 2024-04-16 08:10:22 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:10:22.543Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:10:22 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:10:25.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:10:25.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:10:25.747Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":205,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:10:25.755Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":205,"response":{"results":[{}]}} 2024-04-16T08:10:25.755Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:10:55.7467049 +0000 UTC m=+239.190020297 2024-04-16T08:10:25.755Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:25 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:10:40.7467049 +0000 UTC m=+224.190020297 2024-04-16T08:10:31.022Z [pebble] GET /v1/notices?timeout=30s 30.001157995s 200 2024-04-16T08:10:40.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:10:40.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:10:40.750Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":206,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:10:40.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":206,"response":{"results":[{}]}} 2024-04-16T08:10:40.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:11:10.750478208 +0000 UTC m=+254.193793592 2024-04-16T08:10:40.757Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:40 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:10:55.750478208 +0000 UTC m=+239.193793592 2024-04-16T08:10:55.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:10:55.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:10:55.759Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":207,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:10:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":207,"response":{"results":[{}]}} 2024-04-16T08:10:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:11:25.759284233 +0000 UTC m=+269.202599620 2024-04-16T08:10:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:10:55 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:11:10.759284233 +0000 UTC m=+254.202599620 2024-04-16T08:10:59.520Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:59 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:10:59.526Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:59 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:10:59.528Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:10:59 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:11:00.097Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:00 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:11:00.102Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:00 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:11:00.112Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:00 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:11:00.499Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:00 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:11:00.897Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:00 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:11:00.900Z [jujud] 2024-04-16 08:11:00 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:11:00.900Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:11:00 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:11:01.024Z [pebble] GET /v1/notices?timeout=30s 30.000954923s 200 2024-04-16T08:11:06.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":21,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:11:06.242Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":21,"response":{}} 2024-04-16T08:11:06.919Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":208,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:11:06.920Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":208,"response":{}} 2024-04-16T08:11:10.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:11:10.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:11:10.762Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":209,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:11:10.771Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":209,"response":{"results":[{}]}} 2024-04-16T08:11:10.771Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:11:40.762373749 +0000 UTC m=+284.205689169 2024-04-16T08:11:10.771Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:11:25.762373749 +0000 UTC m=+269.205689169 2024-04-16T08:11:25.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:11:25.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:11:25.763Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":210,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:11:25.770Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":210,"response":{"results":[{}]}} 2024-04-16T08:11:25.770Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:11:55.762939708 +0000 UTC m=+299.206255114 2024-04-16T08:11:25.770Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:25 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:11:40.762939708 +0000 UTC m=+284.206255114 2024-04-16T08:11:31.024Z [pebble] GET /v1/notices?timeout=30s 30.000209591s 200 2024-04-16T08:11:40.766Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:11:40.766Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:11:40.766Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":211,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:11:40.775Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":211,"response":{"results":[{}]}} 2024-04-16T08:11:40.775Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:12:10.765561684 +0000 UTC m=+314.208877067 2024-04-16T08:11:40.775Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:40 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:11:55.765561684 +0000 UTC m=+299.208877067 2024-04-16T08:11:48.303Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:11:48.309Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for volumes-web-app-viewer-spec-ck6bhh4bdm of /v1, Kind=ConfigMap in namespace test 2024-04-16T08:11:48.310Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:11:48.480Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes of /v1, Kind=ServiceAccount in namespace test 2024-04-16T08:11:48.490Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRole in namespace 2024-04-16T08:11:48.514Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-test-kubeflow-volumes of rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding in namespace 2024-04-16T08:11:48.881Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:48 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:11:49.682Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:11:49 DEBUG juju.worker.caasadmission handler.go:113 received admission request for kubeflow-volumes-volumes-web-app-secret of /v1, Kind=Secret in namespace test 2024-04-16T08:11:49.690Z [jujud] 2024-04-16 08:11:49 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:11:49.690Z [jujud] a0a72c11-233c-4cb9-898f-69d9b2f0537c: controller-0 2024-04-16 08:11:49 ERROR juju.worker.dependency engine.go:695 "caas-unit-provisioner" manifold worker returned unexpected error: panic resulted in: assignment to entry in nil map 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.remotestate watcher.go:809 update status timer triggered for kubeflow-volumes/2 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.upgradeseries resolver.go:48 localState.Kind="continue", localState.UpgradeMachineStatus="not started", remoteState.UpgradeMachineStatus="not started" 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:344 create relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.240Z [jujud] } 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.240Z [jujud] } 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:347 no create relation operation to run 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:49 checking leadership status 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:84 leadership status is up-to-date 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:61 actions are blocked=true; outdated remote charm=true - have pending actions: [] 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:63 actions are blocked, no in flight actions 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 resolver.go:216 no operations in progress; waiting for changes 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:56 relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.240Z [jujud] } 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.240Z [jujud] } 2024-04-16T08:11:50.240Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:59 no relation operation to run 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:132 running op: run update-status hook 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:85 running operation run update-status hook for kubeflow-volumes/2 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:88 acquiring machine lock for kubeflow-volumes/2 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.machinelock machinelock.go:166 acquire machine lock "machine-lock" for kubeflow-volumes/2 uniter (run update-status hook) 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.machinelock machinelock.go:176 machine lock "machine-lock" acquired for kubeflow-volumes/2 uniter (run update-status hook) 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 preparing operation "run update-status hook" for kubeflow-volumes/2 2024-04-16T08:11:50.241Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":212,"type":"PayloadsHookContext","version":1,"request":"List","params":{"entities":null}} 2024-04-16T08:11:50.242Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":212,"response":{"results":null}} 2024-04-16T08:11:50.242Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":213,"type":"Uniter","version":19,"request":"APIAddresses","params":{}} 2024-04-16T08:11:50.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":213,"response":{"result":["10.152.183.39:17070","controller-service.controller-uk8s.svc.cluster.local:17070"]}} 2024-04-16T08:11:50.244Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":214,"type":"Uniter","version":19,"request":"SLALevel","params":{}} 2024-04-16T08:11:50.245Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":214,"response":{"result":"unsupported"}} 2024-04-16T08:11:50.245Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":215,"type":"Uniter","version":19,"request":"CloudAPIVersion","params":{}} 2024-04-16T08:11:50.256Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":215,"response":{"result":"1.25.0"}} 2024-04-16T08:11:50.256Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":216,"type":"Uniter","version":19,"request":"ModelConfig","params":{}} 2024-04-16T08:11:50.258Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":216,"response":{"config":{"agent-metadata-url":"","agent-stream":"released","agent-version":"3.4.2","apt-ftp-proxy":"","apt-http-proxy":"","apt-https-proxy":"","apt-mirror":"","apt-no-proxy":"","authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+fpWf9MRtHeATMLuy6Oa/AsylRV7jzcYSq9uwuskhhV0cnIWW9a2jBQkT7iCNT8fkacoJbnxUNJXXPj/S8LakF3adDkl2SuIZ8ZUvHA99Lse/ZKoNJIjfbIBw1WRUC+wVZIDVEqtRd9/jPV0+3YMdf1aX3NjeEmrBtupnbGdZ58PPyE4sf0hTiuCzQ+tjMkQ3C6gRAwgA3KFNvaWkOzES+ny6fuqn+DD6qe+MD1G1n3J3DRLFL5BjLDWW7WaRm8WPZSEiLsDADPGah7Lvw2E4KYAD1Mr/73OEgpwf70Qg3oAMUSBTihA0xcU1qSY4DmUULQKQo0x3lUzPSNxY8B/ur4uBYvefwrGGVwS2YldYQn6LtcaCo/Ogl4DPGoVaAiz18hddQNaOIt95bJ5mvtJinnOcXQIsFSbvLUmBly7p8/fREpK/C/kHxm8PlWHA+WUEbUjEIJ+Twj399IFQjyMiCxaQWWW+I5xnTu5txInyL0qGht9QZ2nN56xCOiEWrJ5CknqTHNIJPVmxS+tXNP0yZKC80IB9QWonef0nu3MXwHyhXjBodsp1yWeFbD6fbnY3E7UJrGScMlKrSw7FdsT4g6tLdUcxZs44/+qYR/1By5Vid3/ED1EvnnD20kcw8hL0ZmpfcRjdVyALqx5dBAcjhBjPiRcHE0nn/f9/m+KpHw== juju-client-key\n","automatically-retry-hooks":true,"backup-dir":"","charmhub-url":"https://api.charmhub.io","cloudinit-userdata":"","container-image-metadata-defaults-disabled":false,"container-image-metadata-url":"","container-image-stream":"released","container-inherit-properties":"","container-networking-method":"","default-base":"","default-space":"","development":false,"disable-network-management":false,"disable-telemetry":false,"egress-subnets":"","enable-os-refresh-update":true,"enable-os-upgrade":true,"fan-config":"","firewall-mode":"instance","ftp-proxy":"","http-proxy":"","https-proxy":"","ignore-machine-addresses":false,"image-metadata-defaults-disabled":false,"image-metadata-url":"","image-stream":"released","juju-ftp-proxy":"","juju-http-proxy":"","juju-https-proxy":"","juju-no-proxy":"127.0.0.1,localhost,::1","logforward-enabled":false,"logging-config":"\u003croot\u003e=TRACE;unit=TRACE","logging-output":"","lxd-snap-channel":"5.0/stable","max-action-results-age":"336h","max-action-results-size":"5G","max-status-history-age":"336h","max-status-history-size":"5G","mode":"requires-prompts","name":"test","net-bond-reconfigure-delay":17,"no-proxy":"127.0.0.1,localhost,::1","num-container-provision-workers":4,"num-provision-workers":16,"operator-storage":"microk8s-hostpath","provisioner-harvest-mode":"destroyed","proxy-ssh":false,"resource-tags":{},"saas-ingress-allow":"0.0.0.0/0,::/0","secret-backend":"auto","snap-http-proxy":"","snap-https-proxy":"","snap-store-assertions":"","snap-store-proxy":"","snap-store-proxy-url":"","ssh-allow":"0.0.0.0/0,::/0","ssl-hostname-verification":true,"storage-default-block-source":"kubernetes","storage-default-filesystem-source":"kubernetes","test-mode":false,"transmit-vendor-metrics":true,"type":"kubernetes","update-status-hook-interval":"5m","uuid":"946658c9-f947-48cb-89e1-cfc55ab360ac","workload-storage":"microk8s-hostpath"}}} 2024-04-16T08:11:50.258Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":217,"type":"Uniter","version":19,"request":"GetMeterStatus","params":{"entities":[{"tag":"unit-kubeflow-volumes-2"}]}} 2024-04-16T08:11:50.261Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":217,"response":{"results":[{"code":"AMBER","info":"not set"}]}} 2024-04-16T08:11:50.261Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":218,"type":"Uniter","version":19,"request":"OpenedPortRangesByEndpoint","params":{}} 2024-04-16T08:11:50.261Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":218,"response":{"results":[{"error":{"message":"getting opened port ranges for \"application\" not supported","code":"not supported"},"unit-port-ranges":null}]}} 2024-04-16T08:11:50.261Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":219,"type":"SecretsManager","version":2,"request":"GetSecretMetadata","params":{}} 2024-04-16T08:11:50.262Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":219,"response":{"results":null}} 2024-04-16T08:11:50.262Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":220,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: pending\nhook:\n kind: update-status\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:11:50.271Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":220,"response":{"results":[{}]}} 2024-04-16T08:11:50.271Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 executing operation "run update-status hook" for kubeflow-volumes/2 2024-04-16T08:11:50.271Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.uniter.runner runner.go:719 starting jujuc server {unix @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket } 2024-04-16T08:11:50.271Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.juju.sockets sockets.go:65 ignoring error on removing "@/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket": remove @/var/lib/juju/agents/unit-kubeflow-volumes-2/agent.socket: no such file or directory 2024-04-16T08:11:50.428Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "ops 2.12.0 up and running."] 2024-04-16T08:11:50.428Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-update-status-2311942280354053930 2024-04-16T08:11:50.428Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-update-status-2311942280354053930"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:11:50.428Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 ops 2.12.0 up and running. 2024-04-16T08:11:50.452Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:221 running hook tool "juju-log" ["--log-level" "DEBUG" "--" "Emitting Juju event update_status."] 2024-04-16T08:11:50.452Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG jujuc server.go:222 running hook tool "juju-log" for kubeflow-volumes/2-update-status-2311942280354053930 2024-04-16T08:11:50.452Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-update-status-2311942280354053930"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:11:50.452Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG unit.kubeflow-volumes/2.juju-log server.go:325 Emitting Juju event update_status. 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:221 running hook tool "is-leader" ["--format=json"] 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG jujuc server.go:222 running hook tool "is-leader" for kubeflow-volumes/2-update-status-2311942280354053930 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE jujuc server.go:223 hook context id "kubeflow-volumes/2-update-status-2311942280354053930"; dir "/var/lib/juju/agents/unit-kubeflow-volumes-2/charm" 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.leadership tracker.go:156 kubeflow-volumes/2 got claim request for kubeflow-volumes leadership 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.leadership tracker.go:271 resolving kubeflow-volumes leadership ticket for kubeflow-volumes/2... 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.leadership tracker.go:262 kubeflow-volumes/2 still has kubeflow-volumes leadership 2024-04-16T08:11:50.474Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.leadership tracker.go:279 confirming kubeflow-volumes leadership for kubeflow-volumes/2 2024-04-16T08:11:50.702Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.context context.go:1490 checking for reboot request 2024-04-16T08:11:50.702Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch) 2024-04-16T08:11:50.702Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":221,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: run-hook\nopstep: done\nhook:\n kind: update-status\nhook-step: done\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:11:50.712Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":221,"response":{"results":[{}]}} 2024-04-16T08:11:50.712Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:135 committing operation "run update-status hook" for kubeflow-volumes/2 2024-04-16T08:11:50.713Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":222,"type":"Uniter","version":19,"request":"SetState","params":{"args":[{"tag":"unit-kubeflow-volumes-2","uniter-state":"leader: true\nstarted: true\nstopped: false\ninstalled: true\nremoved: false\nstatus-set: true\nop: continue\nopstep: pending\nconfig-hash: fcdd5f6de8ff8978d51226e13416a36fe1bff7e3a6439299b5e38326abc2b015\ntrust-hash: 460878b28bec0c166e1350598ec43ba6978bfb38ee99dbf110fc4be24407d99e\n"}]}} 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":222,"response":{"results":[{}]}} 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.machinelock machinelock.go:206 created rotating log file "/var/log/juju/machine-lock.log" with max size 10 MB and max backups 5 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.machinelock machinelock.go:190 machine lock "machine-lock" released for kubeflow-volumes/2 uniter (run update-status hook) 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.operation executor.go:124 lock released for kubeflow-volumes/2 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.resolver loop.go:242 charmdir: available=true opState: started=true stopped=false changing=false 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.upgradeseries resolver.go:48 localState.Kind="continue", localState.UpgradeMachineStatus="not started", remoteState.UpgradeMachineStatus="not started" 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:344 create relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.719Z [jujud] } 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.719Z [jujud] } 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:347 no create relation operation to run 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:49 checking leadership status 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.leadership resolver.go:84 leadership status is up-to-date 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:61 actions are blocked=true; outdated remote charm=true - have pending actions: [] 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2.actions resolver.go:63 actions are blocked, no in flight actions 2024-04-16T08:11:50.719Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 DEBUG juju.worker.caasoperator.uniter.kubeflow-volumes/2 resolver.go:216 no operations in progress; waiting for changes 2024-04-16T08:11:50.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:56 relation resolver next op for new remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.720Z [jujud] } 2024-04-16T08:11:50.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation statetracker.go:236 "kubeflow-volumes/2" synchronise scopes for remote relations map[int]remotestate.RelationSnapshot{ 2024-04-16T08:11:50.720Z [jujud] } 2024-04-16T08:11:50.720Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:50 TRACE juju.worker.caasoperator.uniter.kubeflow-volumes/2.relation resolver.go:59 no relation operation to run 2024-04-16T08:11:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:11:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:11:55.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":223,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:11:55.775Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":223,"response":{"results":[{}]}} 2024-04-16T08:11:55.775Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:12:25.766608152 +0000 UTC m=+329.209923572 2024-04-16T08:11:55.776Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:11:55 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:12:10.766608152 +0000 UTC m=+314.209923572 2024-04-16T08:12:01.025Z [pebble] GET /v1/notices?timeout=30s 30.000255611s 200 2024-04-16T08:12:06.243Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:12:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":22,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:12:06.243Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: model-946658c9-f947-48cb-89e1-cfc55ab360ac 2024-04-16 08:12:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":22,"response":{}} 2024-04-16T08:12:06.921Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:06 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":224,"type":"Pinger","version":1,"request":"Ping","params":{}} 2024-04-16T08:12:06.921Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:06 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":224,"response":{}} 2024-04-16T08:12:10.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.worker.leadership tracker.go:150 kubeflow-volumes/2 renewing lease for kubeflow-volumes leadership 2024-04-16T08:12:10.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.worker.leadership tracker.go:177 checking kubeflow-volumes/2 for kubeflow-volumes leadership 2024-04-16T08:12:10.767Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.rpc.jsoncodec codec.go:226 -> {"request-id":225,"type":"LeadershipService","version":2,"request":"ClaimLeadership","params":{"params":[{"application-tag":"application-kubeflow-volumes","unit-tag":"unit-kubeflow-volumes-2","duration":30}]}} 2024-04-16T08:12:10.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.rpc.jsoncodec codec.go:120 <- {"request-id":225,"response":{"results":[{}]}} 2024-04-16T08:12:10.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.worker.leadership tracker.go:196 kubeflow-volumes/2 confirmed for kubeflow-volumes leadership until 2024-04-16 08:12:40.767065445 +0000 UTC m=+344.210380827 2024-04-16T08:12:10.777Z [jujud] 946658c9-f947-48cb-89e1-cfc55ab360ac: application-kubeflow-volumes 2024-04-16 08:12:10 TRACE juju.worker.leadership tracker.go:198 kubeflow-volumes/2 will renew kubeflow-volumes leadership at 2024-04-16 08:12:25.767065445 +0000 UTC m=+329.210380827