Service plugin queues should be consumed by all RPC workers

[Eugene Nikanorov]

Currently only parent neutron process consumes messages from service queues such as l3-plugin.
In case of DVR with many L3 agents on computes that can quickly lead to different issues because there is not enough speed of RPC requests processing.

That is actually a problem of other service plugins too: metering, lbaas, fwaas, vpnaas.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/226686

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/228933

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/229325

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/226686
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5be613490deebf494f2ecbbcd8cb5dd0d6f5e1b3
Submitter: Jenkins
Branch: master

commit 5be613490deebf494f2ecbbcd8cb5dd0d6f5e1b3
Author: Eugene Nikanorov <email address hidden>
Date: Wed Sep 23 14:06:54 2015 +0400

    Consume service plugins queues in RPC workers.

    This patch adds all RPC workers to consumers of service
    plugins queues such as metering and l3-plugin.
    This is important for DVR-enabled deployments with hundreds
    of agents.

    Change-Id: I6fea7f409c91b25d2c35b038d6100fdfa85d1905
    Closes-Bug: #1498844

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229325
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=bda2c484cb25d3a02d9beceab83873bd60569e46
Submitter: Jenkins
Branch: master

commit bda2c484cb25d3a02d9beceab83873bd60569e46
Author: Elena Ezhova <email address hidden>
Date: Wed Sep 30 12:36:32 2015 +0300

    Consume FWaaS plugin queue in RPC workers

    This patch adds all RPC workers to consumers of FWaaS plugin queue.
    This is important for DVR-enabled deployments with hundreds
    of agents.

    Change-Id: I656ef6248379c845c4c7862fabd26eedd8b7fd46
    Depends-On: I6fea7f409c91b25d2c35b038d6100fdfa85d1905
    Related-Bug: #1498844

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/238745

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228933
Committed: https://git.openstack.org/cgit/openstack/neutron-lbaas/commit/?id=746b1d5516543c2c4701448529f5246a2be02384
Submitter: Jenkins
Branch: master

commit 746b1d5516543c2c4701448529f5246a2be02384
Author: Elena Ezhova <email address hidden>
Date: Tue Sep 29 17:18:51 2015 +0300

    Consume LBaaS v2 plugin queue in RPC workers

    This patch adds all RPC workers to consumers of LBaaS v2 plugin
    queue.
    This is important for DVR-enabled deployments with hundreds
    of agents.

    Change-Id: Iaec2f623cbbba99d17dc15802ed70603db37e0fb
    Related-Bug: #1498844

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/liberty
Review: https://review.openstack.org/259539

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/238745
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=de5bdc9cc2cde9100802d68309fa6dfe3ef31f4d
Submitter: Jenkins
Branch: stable/liberty

commit de5bdc9cc2cde9100802d68309fa6dfe3ef31f4d
Author: Eugene Nikanorov <email address hidden>
Date: Wed Sep 23 14:06:54 2015 +0400

    Consume service plugins queues in RPC workers.

    This patch adds all RPC workers to consumers of service
    plugins queues such as metering and l3-plugin.
    This is important for DVR-enabled deployments with hundreds
    of agents.

    Closes-Bug: #1498844
    (cherry picked from commit 5be613490deebf494f2ecbbcd8cb5dd0d6f5e1b3)

    ===

    Also include the following fix that keeps backwards compatibility with
    plugins that don't implement start_rpc_listeners method:

    Check if plugin supports starting rpc listeners

    When neutron starts an rpc worker, it checks if the plugin has the method
    "start_rpc_listeners". Since most plugins inherit from a base class, and
    that base class implements the start_rpc_listeners method and raises
    NotImplementedError, the rpc worker will attempt to call that method.
    It should just catch the NotImplementedError and continue on.

    Change-Id: Ie1830b6140acffffd0f283a0d8eefa52067f7650
    Closes-Bug: 1551542
    (cherry picked from commit cd7be292a8d1fb4ce913c4e7fbfc55fda033ffbe)

    ===

    Change-Id: I6fea7f409c91b25d2c35b038d6100fdfa85d1905

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/259539
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=d578ac729c62f370eaf563d8619d4c3f5f071887
Submitter: Jenkins
Branch: stable/liberty

commit d578ac729c62f370eaf563d8619d4c3f5f071887
Author: Elena Ezhova <email address hidden>
Date: Wed Sep 30 12:36:32 2015 +0300

    Consume FWaaS plugin queue in RPC workers

    This patch adds all RPC workers to consumers of FWaaS plugin queue.
    This is important for DVR-enabled deployments with hundreds
    of agents.

    Change-Id: I656ef6248379c845c4c7862fabd26eedd8b7fd46
    Depends-On: I6fea7f409c91b25d2c35b038d6100fdfa85d1905
    Related-Bug: #1498844
    (cherry picked from commit bda2c484cb25d3a02d9beceab83873bd60569e46)

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 7.1.0 release.