[OVN] ovn_hash_ring table cleanup

[Max]

The ovn hash ring implementation is currently based on the hostname.
Does it make sense to implement a periodic cleanup job? E.g. clean offline nodes which are older than 7 days or so.

In our environment, the ovn_hash_ring table grows a lot because our API runs containerized with random hostnames (e.g. neutron-api-5fc8ffb45-dl86p). This means with every api restart we have 64 new entries due to 64 api_workers.

MariaDB [neutron]> select count(*) from ovn_hash_ring;
+----------+
| count(*) |
+----------+
| 80957 |
+----------+
1 row in set (0.003 sec)

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Max:

What version of Neutron API are you using?

Since [1], the Neutron API deletes the "ovn_hash_ring" associated register when the Neutron API is stopped [2]. Please check that you have this in your code. If you have this patch in your code, please check that you stop the container gracefully, allowing the signal.signal(signal.SIGTERM) method to remove the DB register.

The same method is also removing the associated nodes during the transient start period. But according to your report, you are using different random hostnames (that initially was not considered).

Lucas is also going to propose a maintenance task to perform a periodic clean-up, considering the case you are proposing.

Regards.

[1]https://review.opendev.org/q/I4b7376cf7df45fcc6e487970b068d06b4e74e319
[2]https://review.opendev.org/c/openstack/neutron/+/886279/2/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#299

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/893030

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/893140

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/893141

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/893142

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/893143

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/893144

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893030
Committed: https://opendev.org/openstack/neutron/commit/7f777c223e557f7bb0e0290090019eaa60095b50
Submitter: "Zuul (22348)"
Branch: master

commit 7f777c223e557f7bb0e0290090019eaa60095b50
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893140
Committed: https://opendev.org/openstack/neutron/commit/adfe0659c619ad82761c9622bdcfdffed09248da
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit adfe0659c619ad82761c9622bdcfdffed09248da
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 7f777c223e557f7bb0e0290090019eaa60095b50)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893141
Committed: https://opendev.org/openstack/neutron/commit/850da2af847742e483c70c1cb36b11154e5e141a
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 850da2af847742e483c70c1cb36b11154e5e141a
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 7f777c223e557f7bb0e0290090019eaa60095b50)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893142
Committed: https://opendev.org/openstack/neutron/commit/79010935c0441598c2ce6d82f59d1fb828f3437e
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 79010935c0441598c2ce6d82f59d1fb828f3437e
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 7f777c223e557f7bb0e0290090019eaa60095b50)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893144
Committed: https://opendev.org/openstack/neutron/commit/ba6c2eeeb7c6573ff5d33b459e89adaccccfa8ac
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit ba6c2eeeb7c6573ff5d33b459e89adaccccfa8ac
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 7f777c223e557f7bb0e0290090019eaa60095b50)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/893143
Committed: https://opendev.org/openstack/neutron/commit/98a1476ed88027ea6b88515163428fae9663c776
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 98a1476ed88027ea6b88515163428fae9663c776
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Aug 29 11:37:52 2023 +0100

    [OVN] Cleanup old Hash Ring node entries

    This patch introduces a maintenance task that runs once a day and is
    responsible for cleaning up Hash Ring nodes that haven't been updated in
    5 days or more.

    Change-Id: Ibed9e0d77500570c3d0f9f39bfe40cb9239d0d7a
    Closes-Bug: #2033281
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 7f777c223e557f7bb0e0290090019eaa60095b50)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 23.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 22.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 20.5.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 21.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron wallaby-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron xena-eom release.