Failed creating instance record: Add instance info to the database: This instance already exists

[Erik Lönroth]

When I'm deploying to a LXD container on a machine on Google Cloud. It fails with the message:

# Deployed a charm, spawning 3 machines and some lxd and kvm containers

$ juju deploy corehooks-all -n 3

$ juju deploy mysql --to kvm:0

$ juju add-unit mysql --to lxd:1

# Tried to deploy
$ juju add-unit mysql --to lxd:2

$ juju debug-log

machine-2: 11:58:35 WARNING juju.worker.provisioner failed to start machine 2/lxd/0 (Failed creating instance record: Add instance info to the database: This instance already exists), retrying in 10s (1 more attempts)
machine-2: 11:58:46 WARNING juju.container.broker no name servers supplied by provider, using host's name servers.
machine-2: 11:58:46 WARNING juju.container.broker no search domains supplied by provider, using host's search domains.
machine-2: 11:58:46 WARNING juju.container.broker incomplete DNS config found, discovering host's DNS config
machine-2: 11:58:46 ERROR juju.worker.provisioner cannot start instance for machine "2/lxd/0": Failed creating instance record: Add instance info to the database: This instance already exists

See also the screenshot on the juju status.

juju version: 2.9.18-ubuntu-amd64

$ juju show-controller
jaas:
  details:
    uuid: a030379a-940f-4760-8fcf-3062b41a04e7
    controller-uuid: a030379a-940f-4760-8fcf-3062b41a04e7
    api-endpoints: ['jimm.jujucharms.com:443']
    cloud: ""
    agent-version: 2.9.18
    agent-git-commit: f51ff05df1bd5deb3f0cf8fd2774ab04fc8d5090
    controller-model-version: (unauthorized access)
    identity-url: https://api.jujucharms.com/identity
    ca-cert: ""
  current-model: erik-lonroth@external/gexample
  account:
    user: erik-lonroth@external
    access: (not implemented)

[Erik Lönroth (erik-lonroth)]

Possibly related: https://bugs.launchpad.net/bugs/1945813

[Heather Lanigan (hmlanigan)]

@erik-lonroth, this does appear to be a duplicate of 1945813. Thank you for reporting. With the simpler reproducer, it'll be much easier to debug.

[Heather Lanigan (hmlanigan)]

This may be a different problem with the same final error message. Investigation shows the initial failure to be:

2021-12-08 21:49:32 WARNING juju.worker.provisioner provisioner_task.go:1114 machine 1/lxd/0 failed to start: websocket: close 1006 (abnormal closure): unexpected EOF
2021-12-08 21:49:32 WARNING juju.worker.provisioner provisioner_task.go:1153 failed to start machine 1/lxd/0 (websocket: close 1006 (abnormal closure): unexpected EOF), retrying in 10s (10 more attempts)

In 1945813, this does not occur, as far as I've seen in the logs. As well, it's 1 out of 7 containers which have this error.

I have found that the machines created have only 1G of memory. If the lxd error does not occur, they fail to have network capability. The error is no longer seen if the machines are created with a memory constraint of 4G

Investigating if a minimum mem of 1G for non controller machines is still an appropriate value.