[OVN] Stop using neutron.api.rpc.handlers.resources_rpc with OVN as a backend

Bug #1889737 reported by Maciej Jozefczyk
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
yatin

Bug Description

I noticed that in devstack master we have a lot of logs like:

Jul 21 08:18:13.371897 ubuntu-bionic-rax-iad-0018525571 neutron-server[6599]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-968a9155-c80a-4fbc-9c10-bcfca6ba372d None None] Pushing event updated for resources: {'Port': ['ID=fedef62e-0a31-4136-a693-4b4d3bab289e,revision_number=5']} {{(pid=6988) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:243}}

That means we're pushing resources updates via RPC, but it is not needed with OVN as a backend, because there is no consumer of those messages.

Lets try to stop doing it.

Tags: ovn
Revision history for this message
Akihiro Motoki (amotoki) wrote :

Disabling RPC notification with OVN makes sense.
resources_rpc.ResourcesPushRpcApi is instantiated in the ML2 plugin. It looks like we need some mechanism to determine whether RPC notification is required based on mechanism driver(s) currently enabled.
In addition, some service plugins use resources_rpc.ResourcesPushRpcApi too and some of them assume RPC notification. They also need to be clean up.

Changed in neutron:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
yatin (yatinkarel) wrote :

As there can be agents running even with ovn which may be consumer of these messages, i think disabling this with a config option would be safer for the user/operators who wish to disable these and knows they don't need any other agents. Will propose a patch to do so.

Changed in neutron:
assignee: nobody → yatin (yatinkarel)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/823637

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
Ihtisham ul Haq (iulhaq) wrote :

We have similar issue but in our case `neutron.plugins.ml2.plugin.ML2Plugin.start_rpc_listeners()` is being called which fails connect to the message queue because there is none, as we are using OVN as backend and have notification driver disabled:

Neutron config:
...
  [oslo_messaging_notifications]
  driver = noop
...

Logs:
```
DEBUG neutron.plugins.ml2.plugin [req-a525ff71-2ed8-481f-a0ae-86221c918416 - - - - -] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} wrapper /usr/local/lib/python3.10/dist-packages/oslo_log/helpers.py:65
ERROR oslo.messaging._drivers.impl_rabbit [req-a525ff71-2ed8-481f-a0ae-86221c918416 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 1.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
```

So I guess the same fix about can be used to make sure to execute `start_rpc_listeners()` only if we have rpc workers.

Revision history for this message
yatin (yatinkarel) wrote :

@iulhaq i created a seperate bug for this https://bugs.launchpad.net/neutron/+bug/1992352, Can you check that and confirm you see similar behavior?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/823637
Committed: https://opendev.org/openstack/neutron/commit/3e1e2d63b3383d28c9a36b00000ab89caffa3829
Submitter: "Zuul (22348)"
Branch: master

commit 3e1e2d63b3383d28c9a36b00000ab89caffa3829
Author: yatinkarel <email address hidden>
Date: Thu Jan 6 04:49:03 2022 +0000

    Enable rpc notifications only when rpc_workers >= 1

    rpc_workers can be set < 1 with 'ovn' backend when no
    other agent is running apart from ovn agents to
    consume these rpc notifications.

    Add and apply disable_notifications decorator on
    methods which do rpc cast calls to agents, the
    decorator makes the caller method execute only
    when rpc_workers >=1. This patch not changing
    default behavior and utilizes the rpc_workers config option
    to enable rpc notification on resources updates only when
    rpc_workers >= 1.

    Also set rpc_workers=0 in ovn jobs to cover this scenario.

    Closes-Bug: #1889737
    Closes-Bug: #1992352
    Change-Id: I700fe2cd422bc1eb8b5144ec116e7f0a60238419

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/872644
Committed: https://opendev.org/openstack/neutron/commit/47fef55e25fb1b7d3c6c7d6083a916e26fd45b6d
Submitter: "Zuul (22348)"
Branch: master

commit 47fef55e25fb1b7d3c6c7d6083a916e26fd45b6d
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Feb 3 13:11:24 2023 +0100

    Add a method to retrieve and register "rpc_workers" config knob

    This new method retrieves the config option "rpc_workers" from the
    configuration. If this option is not loaded, the method registers
    the ``neutron.conf.service.SERVICE_OPTS`` options before reading
    the knob again.

    Closes-Bug: #2004656
    Related-Bug: #1889737

    Change-Id: I1f99cb32f33cc91141136cb4e3fbd33715530c59

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 22.0.0.0rc1

This issue was fixed in the openstack/neutron 22.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.