MachineSuite.TestDyingMachine failure

Bug #1881979 reported by Ian Booth
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
Low
Unassigned

Bug Description

FAIL: machine_test.go:224: MachineSuite.TestDyingMachine
...
[LOG] 0:04.494 DEBUG juju.rpc error closing codec: write tcp 127.0.0.1:38101->127.0.0.1:46178: write: broken pipe
[LOG] 0:04.494 INFO juju.apiserver.connection agent disconnected: machine-0 for deadbeef-0bad-400d-8000-4b1d0d06f00d
[LOG] 0:04.495 DEBUG juju.apiserver [3] machine-0 API connection terminated after 687.078216ms

agent didn't stop, here's what it was doing

{goroutine profile: total 73
10 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49da2b 0x49da0d 0x61acbf 0x62e67e 0x70bcf0 0x504771 0x70bf3c 0x70a4f5 0x70e61b 0x70e626 0xaf55d3 0xaf5bec 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49da2a internal/poll.(*pollDesc).waitRead+0x19a /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49da0c internal/poll.(*FD).Read+0x17c /snap/go/5830/src/internal/poll/fd_unix.go:169
# 0x61acbe net.(*netFD).Read+0x4e /snap/go/5830/src/net/fd_unix.go:202
# 0x62e67d net.(*conn).Read+0x8d /snap/go/5830/src/net/net.go:184
# 0x70bcef crypto/tls.(*atLeastReader).Read+0x5f /snap/go/5830/src/crypto/tls/conn.go:760
# 0x504770 bytes.(*Buffer).ReadFrom+0xb0 /snap/go/5830/src/bytes/buffer.go:204
# 0x70bf3b crypto/tls.(*Conn).readFromUntil+0xeb /snap/go/5830/src/crypto/tls/conn.go:782
# 0x70a4f4 crypto/tls.(*Conn).readRecordOrCCS+0x114 /snap/go/5830/src/crypto/tls/conn.go:589
# 0x70e61a crypto/tls.(*Conn).readRecord+0x15a /snap/go/5830/src/crypto/tls/conn.go:557
# 0x70e625 crypto/tls.(*Conn).Read+0x165 /snap/go/5830/src/crypto/tls/conn.go:1233
# 0xaf55d2 gopkg.in/mgo%2ev2.fill+0x52 /<email address hidden>+incompatible/socket.go:535
# 0xaf5beb gopkg.in/mgo%2ev2.(*mongoSocket).readLoop+0x54b /<email address hidden>+incompatible/socket.go:551

5 @ 0x43a080 0x44a373 0x89d7f5 0x89c108 0x46a3a1
# 0x89d7f4 github.com/juju/worker/v2.(*Runner).run+0x214 /<email address hidden>/runner.go:346
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

5 @ 0x43a080 0x44a373 0xd076c1 0x46a3a1
# 0xd076c0 github.com/juju/pubsub.(*subscriber).loop+0xd0 /<email address hidden>/subscriber.go:59

3 @ 0x43a080 0x45745a 0xad60d3 0x46a3a1
# 0x457459 time.Sleep+0xb9 /snap/go/5830/src/runtime/time.go:188
# 0xad60d2 gopkg.in/mgo%2ev2.(*mongoServer).pinger+0x462 /<email address hidden>+incompatible/server.go:301

2 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49da2b 0x49da0d 0x4a4e11 0x4a4de4 0x504771 0x47858c 0x584bc3 0x584b93 0x584c47 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49da2a internal/poll.(*pollDesc).waitRead+0x19a /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49da0c internal/poll.(*FD).Read+0x17c /snap/go/5830/src/internal/poll/fd_unix.go:169
# 0x4a4e10 os.(*File).read+0x70 /snap/go/5830/src/os/file_unix.go:263
# 0x4a4de3 os.(*File).Read+0x43 /snap/go/5830/src/os/file.go:116
# 0x504770 bytes.(*Buffer).ReadFrom+0xb0 /snap/go/5830/src/bytes/buffer.go:204
# 0x47858b io.copyBuffer+0x2fb /snap/go/5830/src/io/io.go:391
# 0x584bc2 io.Copy+0x62 /snap/go/5830/src/io/io.go:364
# 0x584b92 os/exec.(*Cmd).writerDescriptor.func1+0x32 /snap/go/5830/src/os/exec/exec.go:311
# 0x584c46 os/exec.(*Cmd).Start.func1+0x26 /snap/go/5830/src/os/exec/exec.go:441

2 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49da2b 0x49da0d 0x61acbf 0x62e67e 0x70bcf0 0x504771 0x70bf3c 0x70a4f5 0x70e61b 0x70e626 0x51c443 0x51c5af 0xa7f8d0 0xa81dfc 0xa82d7f 0xa83b6f 0xb1c6df 0xb1b050 0xb17ffd 0xb17c88 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49da2a internal/poll.(*pollDesc).waitRead+0x19a /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49da0c internal/poll.(*FD).Read+0x17c /snap/go/5830/src/internal/poll/fd_unix.go:169
# 0x61acbe net.(*netFD).Read+0x4e /snap/go/5830/src/net/fd_unix.go:202
# 0x62e67d net.(*conn).Read+0x8d /snap/go/5830/src/net/net.go:184
# 0x70bcef crypto/tls.(*atLeastReader).Read+0x5f /snap/go/5830/src/crypto/tls/conn.go:760
# 0x504770 bytes.(*Buffer).ReadFrom+0xb0 /snap/go/5830/src/bytes/buffer.go:204
# 0x70bf3b crypto/tls.(*Conn).readFromUntil+0xeb /snap/go/5830/src/crypto/tls/conn.go:782
# 0x70a4f4 crypto/tls.(*Conn).readRecordOrCCS+0x114 /snap/go/5830/src/crypto/tls/conn.go:589
# 0x70e61a crypto/tls.(*Conn).readRecord+0x15a /snap/go/5830/src/crypto/tls/conn.go:557
# 0x70e625 crypto/tls.(*Conn).Read+0x165 /snap/go/5830/src/crypto/tls/conn.go:1233
# 0x51c442 bufio.(*Reader).fill+0x102 /snap/go/5830/src/bufio/bufio.go:100
# 0x51c5ae bufio.(*Reader).Peek+0x4e /snap/go/5830/src/bufio/bufio.go:138
# 0xa7f8cf github.com/gorilla/websocket.(*Conn).read+0x3f /<email address hidden>/conn.go:357
# 0xa81dfb github.com/gorilla/websocket.(*Conn).advanceFrame+0x5b /<email address hidden>/conn.go:787
# 0xa82d7e github.com/gorilla/websocket.(*Conn).NextReader+0x8e /<email address hidden>/conn.go:947
# 0xa83b6e github.com/gorilla/websocket.(*Conn).ReadJSON+0x2e /<email address hidden>/json.go:50
# 0xb1c6de github.com/juju/juju/rpc/jsoncodec.(*wsJSONConn).Receive+0xae /home/ubuntu/go/src/github.com/juju/juju/rpc/jsoncodec/conn.go:46
# 0xb1b04f github.com/juju/juju/rpc/jsoncodec.(*Codec).ReadHeader+0x7f /home/ubuntu/go/src/github.com/juju/juju/rpc/jsoncodec/codec.go:120
# 0xb17ffc github.com/juju/juju/rpc.(*Conn).loop+0x9c /home/ubuntu/go/src/github.com/juju/juju/rpc/server.go:390
# 0xb17c87 github.com/juju/juju/rpc.(*Conn).input+0x57 /home/ubuntu/go/src/github.com/juju/juju/rpc/server.go:363

2 @ 0x43a080 0x44a373 0xace8a8 0x46a3a1
# 0xace8a7 gopkg.in/mgo%2ev2.(*mongoCluster).syncServersLoop+0x2d7 /<email address hidden>+incompatible/cluster.go:394

2 @ 0x43a080 0x44a373 0xb259f4 0x46a3a1
# 0xb259f3 github.com/juju/worker/v2/catacomb.Invoke.func2+0x123 /<email address hidden>/catacomb/catacomb.go:102

1 @ 0x40dcb4 0x44f17c 0x574072 0x46a3a1
# 0x44f17b os/signal.signal_recv+0x9b /snap/go/5830/src/runtime/sigqueue.go:147
# 0x574071 os/signal.loop+0x21 /snap/go/5830/src/os/signal/signal_unix.go:23

1 @ 0x43a080 0x408567 0x40822b 0x1f292ef 0x1f2a3d5 0x46a3a1
# 0x1f292ee github.com/juju/juju/worker/fortress.(*fortress).Visit+0x16e /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/fortress.go:60
# 0x1f2a3d4 github.com/juju/juju/worker/fortress.Occupy.func2+0x44 /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/occupy.go:50

1 @ 0x43a080 0x408567 0x40822b 0x204c7d7 0x89c108 0x46a3a1
# 0x204c7d6 github.com/juju/juju/worker/introspection.(*socketListener).run+0xc6 /home/ubuntu/go/src/github.com/juju/juju/worker/introspection/socket.go:135
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x408567 0x40822b 0x2696bbc 0x26d0053 0x89c108 0x46a3a1
# 0x2696bbb github.com/juju/juju/apiserver.(*Server).loop+0x27b /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:520
# 0x26d0052 github.com/juju/juju/apiserver.newServer.func3+0xd2 /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:374
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x408567 0x40822b 0x288cd0d 0x288ccd1 0x2893a0d 0x2893886 0x2893705 0x28abba6 0x35e3f37 0x35e3f1e 0x51701c 0x46a3a1
# 0x288cd0c gopkg.in/check%2ev1.(*suiteRunner).runTest+0x11c /<email address hidden>/check.go:819
# 0x288ccd0 gopkg.in/check%2ev1.(*suiteRunner).run+0xe0 /<email address hidden>/check.go:624
# 0x2893a0c gopkg.in/check%2ev1.Run+0x4c /<email address hidden>/run.go:92
# 0x2893885 gopkg.in/check%2ev1.RunAll+0x95 /<email address hidden>/run.go:84
# 0x2893704 gopkg.in/check%2ev1.TestingT+0x394 /<email address hidden>/run.go:72
# 0x28abba5 github.com/juju/testing.MgoTestPackage+0xc5 /<email address hidden>/mgo.go:559
# 0x35e3f36 github.com/juju/juju/testing.MgoSSLTestPackage+0x36 /home/ubuntu/go/src/github.com/juju/juju/testing/mgo.go:25
# 0x35e3f1d github.com/juju/juju/cmd/jujud/agent.TestPackage+0x1d /home/ubuntu/go/src/github.com/juju/juju/cmd/jujud/agent/package_test.go:23
# 0x51701b testing.tRunner+0xdb /snap/go/5830/src/testing/testing.go:991

1 @ 0x43a080 0x408567 0x40822b 0x51740e 0x51b468 0x51701c 0x518917 0x51786f 0x36082c5 0x439cba 0x46a3a1
# 0x51740d testing.(*T).Run+0x37d /snap/go/5830/src/testing/testing.go:1043
# 0x51b467 testing.runTests.func1+0x77 /snap/go/5830/src/testing/testing.go:1284
# 0x51701b testing.tRunner+0xdb /snap/go/5830/src/testing/testing.go:991
# 0x518916 testing.runTests+0x2a6 /snap/go/5830/src/testing/testing.go:1282
# 0x51786e testing.(*M).Run+0x15e /snap/go/5830/src/testing/testing.go:1199
# 0x36082c4 main.main+0x134 _testmain.go:46
# 0x439cb9 runtime.main+0x1f9 /snap/go/5830/src/runtime/proc.go:203

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x1f2908d 0x8a71c5 0x8a4409 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x1f2908c github.com/juju/juju/worker/fortress.(*fortress).Wait+0x2c /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/fortress.go:38
# 0x8a71c4 github.com/juju/worker/v2/dependency.(*Engine).runWorker.func2+0x374 /<email address hidden>/dependency/engine.go:535
# 0x8a4408 github.com/juju/worker/v2/dependency.(*Engine).runWorker+0xe8 /<email address hidden>/dependency/engine.go:539

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x1f85611 0xb25b93 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x1f85610 github.com/juju/juju/core/cache.(*Controller).Wait+0x30 /home/ubuntu/go/src/github.com/juju/juju/core/cache/controller.go:271
# 0xb25b92 github.com/juju/worker/v2/catacomb.(*Catacomb).add.func1+0x82 /<email address hidden>/catacomb/catacomb.go:176

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x28d0c86 0x28d0c7d 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x28d0c85 github.com/juju/juju/apiserver.(*Server).Wait+0x55 /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:421
# 0x28d0c7c github.com/juju/juju/provider/dummy.(*environ).Bootstrap.func1.7+0x4c /home/ubuntu/go/src/github.com/juju/juju/provider/dummy/environs.go:1021

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x35baecd 0x35baeb7 0x35f0a24 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x35baecc github.com/juju/worker/v2.(*Runner).Wait+0x6ac /<email address hidden>/runner.go:253
# 0x35baeb6 github.com/juju/juju/cmd/jujud/agent.(*MachineAgent).Run+0x696 /home/ubuntu/go/src/github.com/juju/juju/cmd/jujud/agent/machine.go:522
# 0x35f0a23 github.com/juju/juju/cmd/jujud/agent.(*MachineSuite).TestDyingMachine.func1+0x33 /home/ubuntu/go/src/github.com/juju/juju/cmd/jujud/agent/machine_test.go:229

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x8a1ed5 0x35ec55f 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x8a1ed4 github.com/juju/worker/v2/dependency.(*Engine).Wait+0x34 /<email address hidden>/dependency/engine.go:223
# 0x35ec55e github.com/juju/juju/cmd/jujud/agent.startIntrospection.func1+0x2e /home/ubuntu/go/src/github.com/juju/juju/cmd/jujud/agent/introspection.go:70

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0x8a1ed5 0x89f375 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0x8a1ed4 github.com/juju/worker/v2/dependency.(*Engine).Wait+0x34 /<email address hidden>/dependency/engine.go:223
# 0x89f374 github.com/juju/worker/v2.(*Runner).runWorker+0x404 /<email address hidden>/runner.go:544

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0xb28111 0xb280fe 0x1f2a383 0x1f29b3b 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0xb28110 github.com/juju/worker/v2/catacomb.(*Catacomb).Wait+0x30 /<email address hidden>/catacomb/catacomb.go:205
# 0xb280fd github.com/juju/juju/core/watcher.(*StringsWorker).Wait+0x1d /home/ubuntu/go/src/github.com/juju/juju/core/watcher/strings.go:140
# 0x1f2a382 github.com/juju/juju/worker/fortress.Occupy.func1+0xf2 /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/occupy.go:38
# 0x1f29b3a github.com/juju/juju/worker/fortress.guestTicket.complete+0x4a /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/fortress.go:148

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0xb28111 0xb280fe 0x8a71c5 0x8a4409 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0xb28110 github.com/juju/worker/v2/catacomb.(*Catacomb).Wait+0x30 /<email address hidden>/catacomb/catacomb.go:205
# 0xb280fd github.com/juju/juju/core/watcher.(*StringsWorker).Wait+0x1d /home/ubuntu/go/src/github.com/juju/juju/core/watcher/strings.go:140
# 0x8a71c4 github.com/juju/worker/v2/dependency.(*Engine).runWorker.func2+0x374 /<email address hidden>/dependency/engine.go:535
# 0x8a4408 github.com/juju/worker/v2/dependency.(*Engine).runWorker+0xe8 /<email address hidden>/dependency/engine.go:539

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0xdfbe61 0x89f375 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0xdfbe60 github.com/juju/juju/state/watcher.(*HubWatcher).Wait+0x30 /home/ubuntu/go/src/github.com/juju/juju/state/watcher/hubwatcher.go:203
# 0x89f374 github.com/juju/worker/v2.(*Runner).runWorker+0x404 /<email address hidden>/runner.go:544

1 @ 0x43a080 0x408567 0x40822b 0x89bf2a 0xe01641 0x89f375 0x46a3a1
# 0x89bf29 gopkg.in/tomb%2ev2.(*Tomb).Wait+0x49 /<email address hidden>/tomb.go:126
# 0xe01640 github.com/juju/juju/state/watcher.(*TxnWatcher).Wait+0x30 /home/ubuntu/go/src/github.com/juju/juju/state/watcher/txnwatcher.go:177
# 0x89f374 github.com/juju/worker/v2.(*Runner).runWorker+0x404 /<email address hidden>/runner.go:544

1 @ 0x43a080 0x408567 0x40826b 0x28d0983 0x46a3a1
# 0x28d0982 github.com/juju/juju/provider/dummy.init.0.func1+0x52 /home/ubuntu/go/src/github.com/juju/juju/provider/dummy/environs.go:307

1 @ 0x43a080 0x408567 0x40826b 0xbbfe45 0xbc1424 0x46a3a1
# 0xbbfe44 gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun+0x44 /<email address hidden>+incompatible/lumberjack.go:408
# 0xbc1423 gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1+0x33 /<email address hidden>+incompatible/lumberjack.go:423

1 @ 0x43a080 0x408567 0x40826b 0xeed3eb 0x46a3a1
# 0xeed3ea k8s.io/klog.(*loggingT).flushDaemon+0x8a /<email address hidden>/klog.go:1010

1 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49da2b 0x49da0d 0x4a4e11 0x4a4de4 0x478cec 0x51ed54 0x28ac272 0x28b1ce3 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49da2a internal/poll.(*pollDesc).waitRead+0x19a /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49da0c internal/poll.(*FD).Read+0x17c /snap/go/5830/src/internal/poll/fd_unix.go:169
# 0x4a4e10 os.(*File).read+0x70 /snap/go/5830/src/os/file_unix.go:263
# 0x4a4de3 os.(*File).Read+0x43 /snap/go/5830/src/os/file.go:116
# 0x478ceb io.(*multiReader).Read+0xab /snap/go/5830/src/io/multi.go:26
# 0x51ed53 bufio.(*Scanner).Scan+0xa3 /snap/go/5830/src/bufio/scan.go:213
# 0x28ac271 github.com/juju/testing.readLastLines+0xf1 /<email address hidden>/mgo.go:611
# 0x28b1ce2 github.com/juju/testing.(*MgoInstance).run.func1+0x372 /<email address hidden>/mgo.go:371

1 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49f3d4 0x49f3b6 0x61b572 0x63a032 0x638dc4 0x735917 0x7c96fd 0x28bfc0e 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49f3d3 internal/poll.(*pollDesc).waitRead+0x1d3 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49f3b5 internal/poll.(*FD).Accept+0x1b5 /snap/go/5830/src/internal/poll/fd_unix.go:384
# 0x61b571 net.(*netFD).accept+0x41 /snap/go/5830/src/net/fd_unix.go:238
# 0x63a031 net.(*TCPListener).accept+0x31 /snap/go/5830/src/net/tcpsock_posix.go:139
# 0x638dc3 net.(*TCPListener).Accept+0x63 /snap/go/5830/src/net/tcpsock.go:261
# 0x735916 crypto/tls.(*listener).Accept+0x36 /snap/go/5830/src/crypto/tls/tls.go:55
# 0x7c96fc net/http.(*Server).Serve+0x25c /snap/go/5830/src/net/http/server.go:2901
# 0x28bfc0d net/http/httptest.(*Server).goServe.func1+0x6d /snap/go/5830/src/net/http/httptest/server.go:308

1 @ 0x43a080 0x432daa 0x432315 0x49cb95 0x49f3d4 0x49f3b6 0x61b572 0x640ae2 0x63ed94 0x7c96fd 0x204c637 0x46a3a1
# 0x432314 internal/poll.runtime_pollWait+0x54 /snap/go/5830/src/runtime/netpoll.go:203
# 0x49cb94 internal/poll.(*pollDesc).wait+0x44 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:87
# 0x49f3d3 internal/poll.(*pollDesc).waitRead+0x1d3 /snap/go/5830/src/internal/poll/fd_poll_runtime.go:92
# 0x49f3b5 internal/poll.(*FD).Accept+0x1b5 /snap/go/5830/src/internal/poll/fd_unix.go:384
# 0x61b571 net.(*netFD).accept+0x41 /snap/go/5830/src/net/fd_unix.go:238
# 0x640ae1 net.(*UnixListener).accept+0x31 /snap/go/5830/src/net/unixsock_posix.go:162
# 0x63ed93 net.(*UnixListener).Accept+0x63 /snap/go/5830/src/net/unixsock.go:260
# 0x7c96fc net/http.(*Server).Serve+0x25c /snap/go/5830/src/net/http/server.go:2901
# 0x204c636 github.com/juju/juju/worker/introspection.(*socketListener).serve+0x2e6 /home/ubuntu/go/src/github.com/juju/juju/worker/introspection/socket.go:130

1 @ 0x43a080 0x44a373 0x1f83c84 0x89c108 0x46a3a1
# 0x1f83c83 github.com/juju/juju/core/cache.(*Controller).loop+0x133 /home/ubuntu/go/src/github.com/juju/juju/core/cache/controller.go:142
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0x2068d2a 0x46a3a1
# 0x2068d29 github.com/juju/juju/worker/logsender.(*BufferedLogWriter).loop+0x109 /home/ubuntu/go/src/github.com/juju/juju/worker/logsender/bufferedlogwriter.go:116

1 @ 0x43a080 0x44a373 0x2673ce3 0x46a3a1
# 0x2673ce2 github.com/juju/juju/apiserver/stateauthenticator.(*Authenticator).Maintain+0x102 /home/ubuntu/go/src/github.com/juju/juju/apiserver/stateauthenticator/auth.go:63

1 @ 0x43a080 0x44a373 0x2699dbd 0x26d11c7 0x266f8ad 0x269977d 0x7c5e14 0x26d0fef 0x7c5e14 0x266ee51 0x266b3b4 0x266cb83 0x7c9353 0x7c4c1c 0x46a3a1
# 0x2699dbc github.com/juju/juju/apiserver.(*Server).serveConn+0x5cc /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:993
# 0x26d11c6 github.com/juju/juju/apiserver.(*Server).apiHandler.func1+0x1b6 /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:931
# 0x266f8ac github.com/juju/juju/apiserver/websocket.Serve+0x11c /home/ubuntu/go/src/github.com/juju/juju/apiserver/websocket/websocket.go:55
# 0x269977c github.com/juju/juju/apiserver.(*Server).apiHandler+0x20c /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:928
# 0x7c5e13 net/http.HandlerFunc.ServeHTTP+0x43 /snap/go/5830/src/net/http/server.go:2012
# 0x26d0fee github.com/juju/juju/apiserver.(*Server).trackRequests.func1+0x14e /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserver.go:899
# 0x7c5e13 net/http.HandlerFunc.ServeHTTP+0x43 /snap/go/5830/src/net/http/server.go:2012
# 0x266ee50 github.com/juju/juju/apiserver/httpcontext.(*QueryModelHandler).ServeHTTP+0xa0 /home/ubuntu/go/src/github.com/juju/juju/apiserver/httpcontext/model.go:52
# 0x266b3b3 github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP+0x153 /<email address hidden>/mux.go:117
# 0x266cb82 github.com/juju/juju/apiserver/apiserverhttp.(*Mux).ServeHTTP+0x72 /home/ubuntu/go/src/github.com/juju/juju/apiserver/apiserverhttp/mux.go:67
# 0x7c9352 net/http.serverHandler.ServeHTTP+0xa2 /snap/go/5830/src/net/http/server.go:2807
# 0x7c4c1b net/http.(*conn).serve+0x86b /snap/go/5830/src/net/http/server.go:1895

1 @ 0x43a080 0x44a373 0x27231fa 0x2722c28 0xb2581b 0xb25ab8 0x89c108 0x46a3a1
# 0x27231f9 github.com/juju/juju/worker/lease.(*Manager).choose+0x349 /home/ubuntu/go/src/github.com/juju/juju/worker/lease/manager.go:201
# 0x2722c27 github.com/juju/juju/worker/lease.(*Manager).loop+0x157 /home/ubuntu/go/src/github.com/juju/juju/worker/lease/manager.go:187
# 0xb2581a github.com/juju/worker/v2/catacomb.runSafely+0x5a /<email address hidden>/catacomb/catacomb.go:290
# 0xb25ab7 github.com/juju/worker/v2/catacomb.Invoke.func3+0x67 /<email address hidden>/catacomb/catacomb.go:116
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0x2737787 0xb2581b 0xb25ab8 0x89c108 0x46a3a1
# 0x2737786 github.com/juju/juju/worker/modelcache.(*cacheWorker).loop+0x776 /home/ubuntu/go/src/github.com/juju/juju/worker/modelcache/worker.go:270
# 0xb2581a github.com/juju/worker/v2/catacomb.runSafely+0x5a /<email address hidden>/catacomb/catacomb.go:290
# 0xb25ab7 github.com/juju/worker/v2/catacomb.Invoke.func3+0x67 /<email address hidden>/catacomb/catacomb.go:116
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0x2741356 0x2737dd2 0x273ac10 0x46a3a1
# 0x2741355 github.com/juju/juju/worker/multiwatcher.(*Watcher).Next+0x2f5 /home/ubuntu/go/src/github.com/juju/juju/worker/multiwatcher/watcher.go:96
# 0x2737dd1 github.com/juju/juju/worker/modelcache.(*cacheWorker).processWatcher+0x61 /home/ubuntu/go/src/github.com/juju/juju/worker/modelcache/worker.go:325
# 0x273ac0f github.com/juju/juju/worker/modelcache.(*cacheWorker).loop.func2+0x12f /home/ubuntu/go/src/github.com/juju/juju/worker/modelcache/worker.go:262

1 @ 0x43a080 0x44a373 0x2743a2f 0x27431a8 0x89c108 0x46a3a1
# 0x2743a2e github.com/juju/juju/worker/multiwatcher.(*Worker).inner+0x33e /home/ubuntu/go/src/github.com/juju/juju/worker/multiwatcher/worker.go:273
# 0x27431a7 github.com/juju/juju/worker/multiwatcher.(*Worker).loop+0x237 /home/ubuntu/go/src/github.com/juju/juju/worker/multiwatcher/worker.go:213
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0x2743f10 0x274586f 0x46a3a1
# 0x2743f0f github.com/juju/juju/worker/multiwatcher.(*Worker).process+0x15f /home/ubuntu/go/src/github.com/juju/juju/worker/multiwatcher/worker.go:306
# 0x274586e github.com/juju/juju/worker/multiwatcher.(*Worker).inner.func1+0x7e /home/ubuntu/go/src/github.com/juju/juju/worker/multiwatcher/worker.go:264

1 @ 0x43a080 0x44a373 0x288bfc4 0x46a3a1
# 0x288bfc3 gopkg.in/check%2ev1.(*resultTracker)._loopRoutine+0xc3 /<email address hidden>/check.go:470

1 @ 0x43a080 0x44a373 0x28b6657 0x46a3a1
# 0x28b6656 github.com/juju/juju/testing.DumpTestLogsAfter.func1+0xb6 /home/ubuntu/go/src/github.com/juju/juju/testing/base.go:268

1 @ 0x43a080 0x44a373 0x2c9a700 0x46a3a1
# 0x2c9a6ff go.opencensus.io/stats/view.(*worker).start+0xff /<email address hidden>/stats/view/worker.go:154

1 @ 0x43a080 0x44a373 0x8a1abe 0x89c108 0x46a3a1
# 0x8a1abd github.com/juju/worker/v2/dependency.(*Engine).loop+0x25d /<email address hidden>/dependency/engine.go:191
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0xb25ce7 0x46a3a1
# 0xb25ce6 github.com/juju/worker/v2/catacomb.(*Catacomb).add.func2+0xe6 /<email address hidden>/catacomb/catacomb.go:182

1 @ 0x43a080 0x44a373 0xbb7257 0x46a3a1
# 0xbb7256 github.com/juju/juju/api.(*monitor).run+0x136 /home/ubuntu/go/src/github.com/juju/juju/api/monitor.go:31

1 @ 0x43a080 0x44a373 0xdfd807 0xe03e8c 0x89c108 0x46a3a1
# 0xdfd806 github.com/juju/juju/state/watcher.(*HubWatcher).loop+0x3b6 /home/ubuntu/go/src/github.com/juju/juju/state/watcher/hubwatcher.go:401
# 0xe03e8b github.com/juju/juju/state/watcher.newHubWatcher.func1+0x10b /home/ubuntu/go/src/github.com/juju/juju/state/watcher/hubwatcher.go:162
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44a373 0xe01cdc 0xe04083 0x89c108 0x46a3a1
# 0xe01cdb github.com/juju/juju/state/watcher.(*TxnWatcher).loop+0x32b /home/ubuntu/go/src/github.com/juju/juju/state/watcher/txnwatcher.go:234
# 0xe04082 github.com/juju/juju/state/watcher.NewTxnWatcher.func1+0x32 /home/ubuntu/go/src/github.com/juju/juju/state/watcher/txnwatcher.go:157
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x43a080 0x44aeb0 0x44ae9b 0x44ab02 0x4779b4 0x1f29887 0x89c108 0x46a3a1
# 0x44ab01 sync.runtime_Semacquire+0x41 /snap/go/5830/src/runtime/sema.go:56
# 0x4779b3 sync.(*WaitGroup).Wait+0x63 /snap/go/5830/src/sync/waitgroup.go:130
# 0x1f29886 github.com/juju/juju/worker/fortress.(*fortress).loop+0x356 /home/ubuntu/go/src/github.com/juju/juju/worker/fortress/fortress.go:87
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x486dd5 0x4ac3b8 0x4a4669 0x582710 0x5826f9 0xa4eb20 0xa4f3f2 0xc30590 0xc24d2a 0xc248f5 0xc2990d 0xc44f35 0x26ff009 0x26fe72f 0x26fe7e0 0x26fbcc8 0xb27e97 0xb27b63 0xb2581b 0xb25ab8 0x89c108 0x46a3a1
# 0x486dd4 syscall.Syscall6+0x4 /snap/go/5830/src/syscall/asm_linux_amd64.s:41
# 0x4ac3b7 os.(*Process).blockUntilWaitable+0x97 /snap/go/5830/src/os/wait_waitid.go:31
# 0x4a4668 os.(*Process).wait+0x38 /snap/go/5830/src/os/exec_unix.go:22
# 0x58270f os.(*Process).Wait+0x5f /snap/go/5830/src/os/exec.go:125
# 0x5826f8 os/exec.(*Cmd).Wait+0x48 /snap/go/5830/src/os/exec/exec.go:507
# 0xa4eb1f github.com/juju/utils/exec.(*RunParams).Wait+0x4f /<email address hidden>/exec/exec.go:192
# 0xa4f3f1 github.com/juju/utils/exec.RunCommands+0x71 /<email address hidden>/exec/exec.go:277
# 0xc3058f github.com/juju/juju/service/systemd.glob..func1+0x5f /home/ubuntu/go/src/github.com/juju/juju/service/systemd/cmdline.go:169
# 0xc24d29 github.com/juju/juju/service/systemd.Cmdline.runCommand+0x79 /home/ubuntu/go/src/github.com/juju/juju/service/systemd/cmdline.go:149
# 0xc248f4 github.com/juju/juju/service/systemd.Cmdline.ListAll+0xc4 /home/ubuntu/go/src/github.com/juju/juju/service/systemd/cmdline.go:110
# 0xc2990c github.com/juju/juju/service/systemd.ListServices+0x2c /home/ubuntu/go/src/github.com/juju/juju/service/systemd/service.go:46
# 0xc44f34 github.com/juju/juju/service.glob..func3+0x414 /home/ubuntu/go/src/github.com/juju/juju/service/service.go:185
# 0x26ff008 github.com/juju/juju/worker/deployer.NewSimpleContext.func2+0x28 /home/ubuntu/go/src/github.com/juju/juju/worker/deployer/simple.go:91
# 0x26fe72e github.com/juju/juju/worker/deployer.(*SimpleContext).deployedUnitsInitSystemJobs+0x2e /home/ubuntu/go/src/github.com/juju/juju/worker/deployer/simple.go:249
# 0x26fe7df github.com/juju/juju/worker/deployer.(*SimpleContext).DeployedUnits+0x3f /home/ubuntu/go/src/github.com/juju/juju/worker/deployer/simple.go:257
# 0x26fbcc7 github.com/juju/juju/worker/deployer.(*Deployer).SetUp+0x197 /home/ubuntu/go/src/github.com/juju/juju/worker/deployer/deployer.go:89
# 0xb27e96 github.com/juju/juju/core/watcher.(*StringsWorker).setUp+0x36 /home/ubuntu/go/src/github.com/juju/juju/core/watcher/strings.go:109
# 0xb27b62 github.com/juju/juju/core/watcher.(*StringsWorker).loop+0x62 /home/ubuntu/go/src/github.com/juju/juju/core/watcher/strings.go:86
# 0xb2581a github.com/juju/worker/v2/catacomb.runSafely+0x5a /<email address hidden>/catacomb/catacomb.go:290
# 0xb25ab7 github.com/juju/worker/v2/catacomb.Invoke.func3+0x67 /<email address hidden>/catacomb/catacomb.go:116
# 0x89c107 gopkg.in/tomb%2ev2.(*Tomb).run+0x37 /<email address hidden>/tomb.go:163

1 @ 0x565055 0x564e70 0x561c0a 0x35d7df9 0x4cd33b 0x4cc844 0x2894f98 0x2894778 0x46a3a1
# 0x565054 runtime/pprof.writeRuntimeProfile+0x94 /snap/go/5830/src/runtime/pprof/pprof.go:694
# 0x564e6f runtime/pprof.writeGoroutine+0x9f /snap/go/5830/src/runtime/pprof/pprof.go:656
# 0x561c09 runtime/pprof.(*Profile).WriteTo+0x3d9 /snap/go/5830/src/runtime/pprof/pprof.go:329
# 0x35d7df8 github.com/juju/juju/cmd/jujud/agent.(*MachineSuite).TestDyingMachine+0x468 /home/ubuntu/go/src/github.com/juju/juju/cmd/jujud/agent/machine_test.go:247
# 0x4cd33a reflect.Value.call+0x8aa /snap/go/5830/src/reflect/value.go:460
# 0x4cc843 reflect.Value.Call+0xb3 /snap/go/5830/src/reflect/value.go:321
# 0x2894f97 gopkg.in/check%2ev1.(*suiteRunner).forkTest.func1+0x627 /<email address hidden>/check.go:781
# 0x2894777 gopkg.in/check%2ev1.(*suiteRunner).forkCall.func1+0x97 /<email address hidden>/check.go:675

 %!s(int=0) %!s(bytes.readOp=0)}
machine_test.go:250:
    c.Fatalf("timed out waiting for agent to terminate")
... Error: timed out waiting for agent to terminate

[LOG] 0:23.862 DEBUG juju.worker.dependency "migration-fortress" manifold worker completed successfully
[LOG] 0:23.862 DEBUG juju.worker.dependency "unit-agent-deployer" manifold worker stopped: connection is shut down
[LOG] 0:23.862 DEBUG juju.cmd.jujud engine stopped, stopping introspection
[LOG] 0:23.862 DEBUG juju.worker.introspection stats worker closing listener
[LOG] 0:23.862 DEBUG juju.worker.introspection stats worker serving finished
[LOG] 0:23.862 DEBUG juju.worker.introspection stats worker finished
[LOG] 0:23.862 DEBUG juju.cmd.jujud introspection stopped
[LOG] 0:23.862 DEBUG juju.api RPC connection died
[LOG] 0:23.862 INFO juju.provider.dummy reset model
[LOG] 0:23.862 DEBUG juju.provider.dummy closing httpServer
[LOG] 0:23.862 DEBUG juju.provider.dummy stopping apiServer
[LOG] 0:23.862 DEBUG juju.rpc error closing codec: write tcp 127.0.0.1:38101->127.0.0.1:45932: write: broken pipe
[LOG] 0:23.862 DEBUG juju.apiserver [1] user-admin API connection terminated after 21.402426331s
[LOG] 0:23.868 DEBUG juju.provider.dummy stopping modelCache worker
[LOG] 0:23.868 DEBUG juju.provider.dummy stopping multiWatcherWorker worker
[LOG] 0:23.868 DEBUG juju.provider.dummy closing apiStatePool
[LOG] 0:23.868 DEBUG juju.state closed state without error
[LOG] 0:23.868 DEBUG juju.provider.dummy resetting MgoServer
[LOG] 0:23.878 INFO juju.testing reset successfully reset admin password
[LOG] 0:25.798 INFO juju.testing reset successfully reset admin password

----------------------------------------------------------------------

Ian Booth (wallyworld)
Changed in juju:
milestone: 2.8.1 → 2.8.2
Changed in juju:
milestone: 2.8.2 → 2.8.3
Pen Gale (pengale)
Changed in juju:
milestone: 2.8.4 → none
Revision history for this message
Canonical Juju QA Bot (juju-qa-bot) wrote :

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.

Changed in juju:
importance: High → Low
tags: added: expirebugs-bot
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.