[yoga edge] ovsdb incomplete

[Marian Gasparovic]

In SQA testing yoga-focal we encountered this issue. It happens after vault is initialized but all ovn related units stay in waiting indefinitely.

Link to artifacts

https://oil-jenkins.canonical.com/artifacts/6e1710fd-5f24-477d-ac74-e6cc92944d00/index.html

Everything else deployed normally, no error states.

ovn-central/0 waiting idle 3/lxd/8 10.246.167.15 6641/tcp,6642/tcp 'ovsdb-peer' incomplete
  logrotated/43 active idle 10.246.167.15 Unit is ready.
ovn-central/1 waiting idle 4/lxd/9 10.246.167.141 6641/tcp,6642/tcp 'ovsdb-peer' incomplete
  logrotated/48 active idle 10.246.167.141 Unit is ready.
ovn-central/2* active idle 5/lxd/10 10.246.166.205 6641/tcp,6642/tcp Unit is ready (leader: ovnnb_db, ovnsb_db northd: active)
  logrotated/32 active idle 10.246.166.205 Unit is ready.

nova-compute/2 active idle 2 10.246.164.153 Unit is ready
  ceilometer-agent/5 active idle 10.246.164.153 Unit is ready
  lldpd/4 active idle 10.246.164.153 ready
  ovn-chassis/5 waiting idle 10.246.164.153 'ovsdb' incomplete

All of the OVN chassis were in this state

[Frode Nordahl (fnordahl)]

1) Does it work with a previous version of Juju?

2) Could you please provide the output from ``juju show-unit`` for all vault, ovn-central and ovn-chassis units?

[Alex Kavanagh (ajkavanagh)]

1) There's not any data around whether it works with a previous version of Juju as this is an SQA run for the release of yoga/quincy/22.03 for the 22.04 charms set.
2) We'll try to get data on the units, but these are from completed (failed) runs and thus the models have been reaped.

However, this will be a release blocker so we need to work out what's potentially going on.

[Frode Nordahl (fnordahl)]

This is all fine, but we routinely deploy the charm on MAAS with spaces (several times today) and we've not encountered this issue. So there is nothing we can do with this without more information.

[Launchpad Janitor (janitor)]

[Expired for charm-ovn-chassis because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for charm-ovn-central because there has been no activity for 60 days.]

[Bas de Bruijne (basdbruijne)]

Its not a very frequent bug, we have seen it on both focal and jammy, yoga and antelope, juju 2.9.28, 2.9.42, 2.9.43.

To find the juju show-unit outputs, 1) download the crashdump (https://oil-jenkins.canonical.com/artifacts/fa7a3246-7c2c-4453-af06-5730f6c7c567/generated/generated/openstack/juju-crashdump-openstack-2023-06-14-04.59.03.tar.gz), 2) navigate to the unit of interest, 3) look in the juju-show-unit dir