Activity log for bug #1773117

Date Who What changed Old value New value Message
2018-05-24 08:54:51 MichaelEino bug added bug
2018-05-24 08:57:12 MichaelEino description Just performed a bootstrap to a KVMs on MAAS v2.3.3 added a new model "deployment" installed postgresql-HA then returned to controller model and enable-ha I got this many times: machine-0: 10:46:03 ERROR juju.worker.dependency "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) u@MAAS:~# juju status Model Controller Cloud/Region Version SLA controller jujumaas maaslab 2.3.7 unsupported Machine State DNS Inst id Series AZ Message 0 started x.x.x.42 4s33ky xenial default Deployed 1 started x.x.x.40 efdqxc xenial default Deployed 2 started x.x.x.41 fw3dw7 xenial default Deployed u@MAAS:~# juju controllers --refresh Controller Model User Access Cloud/Region Models Machines HA Version jujumaas* controller admin superuser maaslab 3 6 3 2.3.7 More log: machine-0: 10:49:28 ERROR juju.worker.dependency "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) machine-0: 10:49:28 DEBUG juju.worker.dependency stack trace: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) github.com/juju/juju/rpc/client.go:149: github.com/juju/juju/api/apiclient.go:924: github.com/juju/juju/api/machiner/machine.go:122: github.com/juju/juju/worker/machiner/machiner.go:169: cannot update observed network config machine-2: 10:49:29 DEBUG juju.worker.peergrouper controller machines in state: []string{"0", "1", "2"} machine-0: 10:49:30 DEBUG juju.worker.peergrouper controller machines in state: []string{"0", "1", "2"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170114,"type":"Agent","version":2,"request":"GetEntities","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.249135ms {"request-id":170114,"response":"'body redacted'"} Agent[""].GetEntities machine-0: 10:49:31 WARNING juju.cmd.jujud determining kvm support: INFO: Your CPU does not support KVM extensions KVM acceleration can NOT be used : exit status 1 no kvm containers possible machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170115,"type":"Provisioner","version":5,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.100905ms {"request-id":170115,"response":"'body redacted'"} Provisioner[""].Life machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170116,"type":"Provisioner","version":5,"request":"SetSupportedContainers","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 61.834742ms {"request-id":170116,"response":"'body redacted'"} Provisioner[""].SetSupportedContainers machine-0: 10:49:31 DEBUG juju.worker start "0-container-watcher" machine-0: 10:49:31 INFO juju.worker start "0-container-watcher" machine-0: 10:49:31 DEBUG juju.worker "0-container-watcher" started machine-0: 10:49:31 DEBUG juju.cmd.jujud upgrades done, starting worker "0-container-watcher" machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170117,"type":"Machiner","version":1,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170118,"type":"Provisioner","version":5,"request":"WatchContainers","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 2.5669ms {"request-id":170117,"response":"'body redacted'"} Machiner[""].Life machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 2.609106ms {"request-id":170118,"response":"'body redacted'"} Provisioner[""].WatchContainers machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170119,"type":"Machiner","version":1,"request":"SetProviderNetworkConfig","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170120,"type":"StringsWatcher","version":1,"id":"19876","request":"Next","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.provider.maas opening model "controller". machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 488.675989ms {"request-id":170119,"response":"'body redacted'"} Machiner[""].SetProviderNetworkConfig machine-0: 10:49:31 DEBUG juju.worker killing runner 0xc4203bb040 machine-0: 10:49:31 INFO juju.worker runner is dying machine-0: 10:49:31 DEBUG juju.worker killing "0-container-watcher" machine-0: 10:49:31 DEBUG juju.cmd.jujud stopping so killing worker "0-container-watcher" machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170121,"type":"StringsWatcher","version":1,"id":"19876","request":"Stop","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 352.256µs {"request-id":170121,"response":"'body redacted'"} StringsWatcher["19876"].Stop machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 490.115139ms {"request-id":170120,"error":"watcher has been stopped","error-code":"stopped","response":"'body redacted'"} StringsWatcher["19876"].Next machine-0: 10:49:31 INFO juju.worker stopped "0-container-watcher", err: <nil> machine-0: 10:49:31 DEBUG juju.worker "0-container-watcher" done: <nil> machine-0: 10:49:31 DEBUG juju.worker no restart, removing "0-container-watcher" from known workers machine-0: 10:49:31 DEBUG juju.worker.dependency "unconverted-api-workers" manifold worker stopped: setting controller network config: cannot set controller provider network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found machine-0: 10:49:31 ERROR juju.worker.dependency "unconverted-api-workers" manifold worker returned unexpected error: setting controller network config: cannot set controller provider network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found machine-0: 10:49:31 DEBUG juju.worker.dependency stack trace: cannot get network interfaces of "4s33ky": instance "4s33ky" not found github.com/juju/juju/cmd/jujud/agent/machine.go:793: cannot set controller provider network config github.com/juju/juju/cmd/jujud/agent/machine.go:756: setting controller network config github.com/juju/juju/worker/fortress/occupy.go:63: github.com/juju/juju/cmd/jujud/agent/engine/housing.go:93: machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170122,"type":"Agent","version":2,"request":"ModelConfig","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.536168ms {"request-id":170122,"response":"'body redacted'"} Agent[""].ModelConfig machine-0: 10:49:31 DEBUG juju.worker.dependency "machiner" manifold worker started machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170123,"type":"Machiner","version":1,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.283401ms {"request-id":170123,"response":"'body redacted'"} Machiner[""].Life machine-0: 10:49:31 DEBUG juju.network no lxc bridge addresses to filter for machine machine-0: 10:49:31 DEBUG juju.network "lxdbr0" has addresses [fe80::d01b:3cff:fec6:fe50/64 fe80::1/64] machine-0: 10:49:31 DEBUG juju.network cannot get "virbr0" addresses: route ip+net: no such network interface (ignoring) machine-0: 10:49:31 DEBUG juju.network including address local-machine:127.0.0.1 for machine machine-0: 10:49:31 DEBUG juju.network including address local-cloud:x.x.x.42 for machine machine-0: 10:49:31 DEBUG juju.network including address local-machine:::1 for machine machine-0: 10:49:31 DEBUG juju.network addresses after filtering: [local-machine:127.0.0.1 local-cloud:x.x.x.42 local-machine:::1] machine-0: 10:49:31 INFO juju.worker.machiner setting addresses for "machine-0" to [local-machine:127.0.0.1 local-cloud:x.x.x.42 local-machine:::1] #1731108 might be related! Just performed a bootstrap to a KVMs on MAAS v2.3.3 added a new model "deployment" installed postgresql-HA then returned to controller model and enable-ha I got this many times: machine-0: 10:46:03 ERROR juju.worker.dependency "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) u@MAAS:~# juju status Model Controller Cloud/Region Version SLA controller jujumaas maaslab 2.3.7 unsupported Machine State DNS Inst id Series AZ Message 0 started x.x.x.42 4s33ky xenial default Deployed 1 started x.x.x.40 efdqxc xenial default Deployed 2 started x.x.x.41 fw3dw7 xenial default Deployed u@MAAS:~# juju controllers --refresh Controller Model User Access Cloud/Region Models Machines HA Version jujumaas* controller admin superuser maaslab 3 6 3 2.3.7 More log: machine-0: 10:49:28 ERROR juju.worker.dependency "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) machine-0: 10:49:28 DEBUG juju.worker.dependency stack trace: cannot get network interfaces of "4s33ky": instance "4s33ky" not found (not found) github.com/juju/juju/rpc/client.go:149: github.com/juju/juju/api/apiclient.go:924: github.com/juju/juju/api/machiner/machine.go:122: github.com/juju/juju/worker/machiner/machiner.go:169: cannot update observed network config machine-2: 10:49:29 DEBUG juju.worker.peergrouper controller machines in state: []string{"0", "1", "2"} machine-0: 10:49:30 DEBUG juju.worker.peergrouper controller machines in state: []string{"0", "1", "2"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170114,"type":"Agent","version":2,"request":"GetEntities","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.249135ms {"request-id":170114,"response":"'body redacted'"} Agent[""].GetEntities machine-0: 10:49:31 WARNING juju.cmd.jujud determining kvm support: INFO: Your CPU does not support KVM extensions KVM acceleration can NOT be used : exit status 1 no kvm containers possible machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170115,"type":"Provisioner","version":5,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.100905ms {"request-id":170115,"response":"'body redacted'"} Provisioner[""].Life machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170116,"type":"Provisioner","version":5,"request":"SetSupportedContainers","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 61.834742ms {"request-id":170116,"response":"'body redacted'"} Provisioner[""].SetSupportedContainers machine-0: 10:49:31 DEBUG juju.worker start "0-container-watcher" machine-0: 10:49:31 INFO juju.worker start "0-container-watcher" machine-0: 10:49:31 DEBUG juju.worker "0-container-watcher" started machine-0: 10:49:31 DEBUG juju.cmd.jujud upgrades done, starting worker "0-container-watcher" machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170117,"type":"Machiner","version":1,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170118,"type":"Provisioner","version":5,"request":"WatchContainers","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 2.5669ms {"request-id":170117,"response":"'body redacted'"} Machiner[""].Life machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 2.609106ms {"request-id":170118,"response":"'body redacted'"} Provisioner[""].WatchContainers machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170119,"type":"Machiner","version":1,"request":"SetProviderNetworkConfig","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170120,"type":"StringsWatcher","version":1,"id":"19876","request":"Next","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.provider.maas opening model "controller". machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 488.675989ms {"request-id":170119,"response":"'body redacted'"} Machiner[""].SetProviderNetworkConfig machine-0: 10:49:31 DEBUG juju.worker killing runner 0xc4203bb040 machine-0: 10:49:31 INFO juju.worker runner is dying machine-0: 10:49:31 DEBUG juju.worker killing "0-container-watcher" machine-0: 10:49:31 DEBUG juju.cmd.jujud stopping so killing worker "0-container-watcher" machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170121,"type":"StringsWatcher","version":1,"id":"19876","request":"Stop","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 352.256µs {"request-id":170121,"response":"'body redacted'"} StringsWatcher["19876"].Stop machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 490.115139ms {"request-id":170120,"error":"watcher has been stopped","error-code":"stopped","response":"'body redacted'"} StringsWatcher["19876"].Next machine-0: 10:49:31 INFO juju.worker stopped "0-container-watcher", err: <nil> machine-0: 10:49:31 DEBUG juju.worker "0-container-watcher" done: <nil> machine-0: 10:49:31 DEBUG juju.worker no restart, removing "0-container-watcher" from known workers machine-0: 10:49:31 DEBUG juju.worker.dependency "unconverted-api-workers" manifold worker stopped: setting controller network config: cannot set controller provider network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found machine-0: 10:49:31 ERROR juju.worker.dependency "unconverted-api-workers" manifold worker returned unexpected error: setting controller network config: cannot set controller provider network config: cannot get network interfaces of "4s33ky": instance "4s33ky" not found machine-0: 10:49:31 DEBUG juju.worker.dependency stack trace: cannot get network interfaces of "4s33ky": instance "4s33ky" not found github.com/juju/juju/cmd/jujud/agent/machine.go:793: cannot set controller provider network config github.com/juju/juju/cmd/jujud/agent/machine.go:756: setting controller network config github.com/juju/juju/worker/fortress/occupy.go:63: github.com/juju/juju/cmd/jujud/agent/engine/housing.go:93: machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170122,"type":"Agent","version":2,"request":"ModelConfig","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.536168ms {"request-id":170122,"response":"'body redacted'"} Agent[""].ModelConfig machine-0: 10:49:31 DEBUG juju.worker.dependency "machiner" manifold worker started machine-0: 10:49:31 DEBUG juju.apiserver <- [1] machine-0 {"request-id":170123,"type":"Machiner","version":1,"request":"Life","params":"'params redacted'"} machine-0: 10:49:31 DEBUG juju.apiserver -> [1] machine-0 1.283401ms {"request-id":170123,"response":"'body redacted'"} Machiner[""].Life machine-0: 10:49:31 DEBUG juju.network no lxc bridge addresses to filter for machine machine-0: 10:49:31 DEBUG juju.network "lxdbr0" has addresses [fe80::d01b:3cff:fec6:fe50/64 fe80::1/64] machine-0: 10:49:31 DEBUG juju.network cannot get "virbr0" addresses: route ip+net: no such network interface (ignoring) machine-0: 10:49:31 DEBUG juju.network including address local-machine:127.0.0.1 for machine machine-0: 10:49:31 DEBUG juju.network including address local-cloud:x.x.x.42 for machine machine-0: 10:49:31 DEBUG juju.network including address local-machine:::1 for machine machine-0: 10:49:31 DEBUG juju.network addresses after filtering: [local-machine:127.0.0.1 local-cloud:x.x.x.42 local-machine:::1] machine-0: 10:49:31 INFO juju.worker.machiner setting addresses for "machine-0" to [local-machine:127.0.0.1 local-cloud:x.x.x.42 local-machine:::1]
2018-06-07 16:21:53 John A Meinel juju: importance Undecided Medium
2018-06-07 16:21:53 John A Meinel juju: status New Triaged
2018-06-07 16:22:05 John A Meinel bug task added maas
2018-06-07 16:22:34 John A Meinel summary ERROR cannot update observed network config MAAS temporarily reporting no instance while bringing up a new instance.
2018-06-07 16:31:32 Andres Rodriguez maas: status New Incomplete
2019-09-19 12:05:41 Adam Collard maas: status Incomplete Expired
2019-09-19 12:26:15 Anastasia juju: status Triaged Incomplete
2020-09-01 22:55:09 Ian Booth tags enable-ha kvm maas maas-provider network enable-ha kvm maas-provider network
2022-11-03 16:52:13 Canonical Juju QA Bot juju: importance Medium Low
2022-11-03 16:52:14 Canonical Juju QA Bot tags enable-ha kvm maas-provider network enable-ha expirebugs-bot kvm maas-provider network