OVN IDLs not initialized for all worker types

[Bodo Petermann]

Since neutron commit "Rely on worker count for HashRing caching" (https://opendev.org/openstack/neutron/commit/c4007b0833111a25d24f597161d39ee9ccd37189) post_fork_initialize is only executed for API and maintenance workers. This means for other worker types the OVN IDLs are not initialized anymore.

I'm working on an implementation of VPNaaS for OVN (https://bugs.launchpad.net/neutron/+bug/1905391) and plan to add an OVN VPN agent whose liveness is checked in a similar way to the OVN metadata agent. A PeriodicWorker regularly checks the agent list to find dead agents. But to have the get_agents method include the OVN agents the PeriodicWorker needs the OVN IDLs and the AgentCache. Both were set up in post_fork_initialize, but now they are not.
Secondly our proposed implementation of OVN VPN agent will still make use of rabbitmq RPCs to sync the VPN service settings. The controller implements some RPC methods that may also need access to the AgentCache. So also the RpcWorker might need post_fork_initialize to set up the OVN IDLs and AgentCache.

If I did not overlook something it could be fine to go back to run post_fork_initialize for all worker types. The main intention of the "Rely on worker count for HashRing caching" commit was to only add the API worker to the hash ring and that's still the case because of a respective "if" in post_fork_initialize.

[Terry Wilson (otherwiseguy)]

The problem with doing post_fork_initialize everywhere is that those DB connections are super expensive. Each one maintains an in-memory copy of the OVS DBs, and ML2/OVN itself doesn't use RPC workers at all. If there is a way to let other code opt-in to setting up DB connections as opposed to just enabling it generally for unneeded workers, it'd be nice.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/795474

[Bodo Petermann (bpetermann)]

There changeset [1] tackles it for a specific case (RpcWorker for sriovnicswitch).
But other plugins or extensions may need it too, so my idea would be to add configuration options like enable_ovn_idl_for_allservices_worker and enable_ovn_idl_for_rpc_workers and if they are set, enable the IDL for neutron.service.AllServicesNeutronWorker and neutron.service.RpcWorker respectively. This could be combined with [1], so that it won't be necessary to set enable_ovn_idl_for_rpc_workers explicitly for sriovnicswitch.

[1] https://review.opendev.org/c/openstack/neutron/+/795474

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/795781

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/795781
Committed: https://opendev.org/openstack/neutron/commit/66ac943b6435901c163c3050ad41dfcaf36ac11d
Submitter: "Zuul (22348)"
Branch: master

commit 66ac943b6435901c163c3050ad41dfcaf36ac11d
Author: Bodo Petermann <email address hidden>
Date: Thu Jun 10 15:10:59 2021 +0200

    Config option to enable OVN IDL on other workers

    Add a configuration option to ovn group:
    additional_worker_classes_with_ovn_idl
    to optionally enable OVN IDL connections on additional workers.
    By default the IDL connections are only initialized on API and
    maintenance workers as other worker types normally don't need them.
    But some plugins or extensions might still require IDL connections on
    other workers and now may enable them via configuration, by adding the
    worker class (or its alias from neutron.worker_classes) to
    additional_worker_classes_with_ovn_idl.

    Closes-Bug: #1927977
    Change-Id: Ia0d882220cd281e3c91648e3f32a842d61a5f57d

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Frode Nordahl <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/795474
Reason: Superseded by unconditionally enabling OVN IDL for RPC workers in https://review.opendev.org/c/openstack/neutron/+/800679

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 19.0.0.0rc1 release candidate.