devstack doesn't clean the ovn database on restack

[Gregory Thiemonge]

Using devstack master (e102559f87b3b2afb5ad0aa874f9bfa98c269624) with ml2/ovn, devstack doesn't clean up the ovn database files on restack while it is supposed to do it.

After a bunch of "./unstack.sh; ./clean.sh; ./stack.sh", I have a _lot_ of entries in the ovn db:

$ sudo ovn-nbctl show | grep router | wc -l
505

$ sudo ovn-nbctl show | grep port | wc -l
1352

It triggers some ARP collisions when allocating IP addresses on the public subnet (neutron allocates a unused IP address but it's already allocated in ovn because of a previous deployment)

When initializing OVN, devstack cleans up some files:

https://opendev.org/openstack/devstack/src/branch/master/lib/neutron_plugins/ovn_agent#L569

rm -f $OVN_DATADIR/*.db

with OVN_DATADIR=/opt/stack/data/ovn

but the ovn service from the packages doesn't use /opt/stack/data/ovn but /var/lib/ovn.

Note: when using OVN_BUILD_FROM_SOURCE=True option, the ovn systemd file is modified to set the datadir to /opt/stack/data/ovn directory

What would be an appropriate fix? Cleaning /var/lib/ovn/*db (the path probably depends on the distrib)? Modifying ovn systemd files to use /opt/stack/data/ovn?

[Radosław Piliszek (yoctozepto)]

I suggest cleaning up the correct path.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/devstack/+/806858

[Martin Kopec (mkopec)]

The following bug got reported as a reaction to the bug tracked in this LP:
https://bugs.launchpad.net/devstack/+bug/1932534

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/806858
Committed: https://opendev.org/openstack/devstack/commit/6822ff39444f8ebe01084e0d92bc05ac40db8216
Submitter: "Zuul (22348)"
Branch: master

commit 6822ff39444f8ebe01084e0d92bc05ac40db8216
Author: Gregory Thiemonge <email address hidden>
Date: Wed Sep 1 09:36:31 2021 +0200

    Fix OVN DBs cleanup on startup

    When initializing OVN, clean up the correct database directory when
    using OVN from packages (/var/lib/ovn/ instead of /opt/stack/data/ovn/).
    The /opt/stack/data/ovn location is used only when building OVN from
    sources, so a fresh devstack deployment with OVN packages may already
    have hundreds of existing routers and ports, creating ARP collisions.

    Closes-Bug: #1942201
    Change-Id: Ic90d4f2f9d8aaef825ea3325c0ad8fef2a1c5e39