DHCP agent takes very long time to report when port is provisioned

[Slawek Kaplonski]

DHCP agent sends through rpc to neutron-server messages about ports which are already configured by agent: https://github.com/openstack/neutron/blob/3ad6af76266f466953de08451b922e033b5c8c43/neutron/agent/dhcp/agent.py#L255
It sends them in chunks with 64 ports at max.
Usually this works fine but in some cases when e.g. network with many ports (like 1000 for example) was resynced for some reason in "dhcp_ready_ports" there will be all those 1000 ports from resynced network.
And than new port may be at the end of this set and will be processed (send to neutron-server) after very long time. In our production cloud it could be even more than 15 minutes.
This cause problem with spawning vm which is in errors state due to "Failed to allocate network" error.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/692459

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/692459
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1a686fb401eca1843b81292fc88c13a2e6fd274d
Submitter: Zuul
Branch: master

commit 1a686fb401eca1843b81292fc88c13a2e6fd274d
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 31 23:12:03 2019 +0100

    Switch to use cast method in dhcp_ready_on_ports method

    When DHCP agent reports to the neutron-server which ports are
    ready, it was using call() method from rpc client.
    That caused blocking dhcp agent's dhcp_ready_ports_loop thread which
    blocks to send info about other ready ports if there are any.

    Method call() should be used when RPC caller returns a value to the
    caller but that's not the case here. On neutron server side this
    RPC method is only calling provisioning_complete() method to
    finish provisioning of ports. And is not returning anything.

    So to make sending dhcp ready ports to neutron-server much faster
    this patch switch to use cast() method from rpc client.
    This method don't block to wait for return value from RPC caller.

    Change-Id: Ie119693854aa283b863a1eac2bdae3330c2b6a9d
    Closes-Bug: #1850864

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/694558

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/694559

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/694560

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/694561

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694558
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=69250307ee3654f46efed4f9d81e2a555d635361
Submitter: Zuul
Branch: stable/train

commit 69250307ee3654f46efed4f9d81e2a555d635361
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 31 23:12:03 2019 +0100

    Switch to use cast method in dhcp_ready_on_ports method

    When DHCP agent reports to the neutron-server which ports are
    ready, it was using call() method from rpc client.
    That caused blocking dhcp agent's dhcp_ready_ports_loop thread which
    blocks to send info about other ready ports if there are any.

    Method call() should be used when RPC caller returns a value to the
    caller but that's not the case here. On neutron server side this
    RPC method is only calling provisioning_complete() method to
    finish provisioning of ports. And is not returning anything.

    So to make sending dhcp ready ports to neutron-server much faster
    this patch switch to use cast() method from rpc client.
    This method don't block to wait for return value from RPC caller.

    Change-Id: Ie119693854aa283b863a1eac2bdae3330c2b6a9d
    Closes-Bug: #1850864
    (cherry picked from commit 1a686fb401eca1843b81292fc88c13a2e6fd274d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694561
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0b1aac544bd9dd0f379d6ae37cd30ce07759dbde
Submitter: Zuul
Branch: stable/queens

commit 0b1aac544bd9dd0f379d6ae37cd30ce07759dbde
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 31 23:12:03 2019 +0100

    Switch to use cast method in dhcp_ready_on_ports method

    When DHCP agent reports to the neutron-server which ports are
    ready, it was using call() method from rpc client.
    That caused blocking dhcp agent's dhcp_ready_ports_loop thread which
    blocks to send info about other ready ports if there are any.

    Method call() should be used when RPC caller returns a value to the
    caller but that's not the case here. On neutron server side this
    RPC method is only calling provisioning_complete() method to
    finish provisioning of ports. And is not returning anything.

    So to make sending dhcp ready ports to neutron-server much faster
    this patch switch to use cast() method from rpc client.
    This method don't block to wait for return value from RPC caller.

    Change-Id: Ie119693854aa283b863a1eac2bdae3330c2b6a9d
    Closes-Bug: #1850864
    (cherry picked from commit 1a686fb401eca1843b81292fc88c13a2e6fd274d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694560
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8c2b944dc08377fa84d1264ce1b24fbbada44c52
Submitter: Zuul
Branch: stable/rocky

commit 8c2b944dc08377fa84d1264ce1b24fbbada44c52
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 31 23:12:03 2019 +0100

    Switch to use cast method in dhcp_ready_on_ports method

    When DHCP agent reports to the neutron-server which ports are
    ready, it was using call() method from rpc client.
    That caused blocking dhcp agent's dhcp_ready_ports_loop thread which
    blocks to send info about other ready ports if there are any.

    Method call() should be used when RPC caller returns a value to the
    caller but that's not the case here. On neutron server side this
    RPC method is only calling provisioning_complete() method to
    finish provisioning of ports. And is not returning anything.

    So to make sending dhcp ready ports to neutron-server much faster
    this patch switch to use cast() method from rpc client.
    This method don't block to wait for return value from RPC caller.

    Change-Id: Ie119693854aa283b863a1eac2bdae3330c2b6a9d
    Closes-Bug: #1850864
    (cherry picked from commit 1a686fb401eca1843b81292fc88c13a2e6fd274d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694559
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=36977d5d0ce337bf0bcdeaee49ce6e9723a3810c
Submitter: Zuul
Branch: stable/stein

commit 36977d5d0ce337bf0bcdeaee49ce6e9723a3810c
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 31 23:12:03 2019 +0100

    Switch to use cast method in dhcp_ready_on_ports method

    When DHCP agent reports to the neutron-server which ports are
    ready, it was using call() method from rpc client.
    That caused blocking dhcp agent's dhcp_ready_ports_loop thread which
    blocks to send info about other ready ports if there are any.

    Method call() should be used when RPC caller returns a value to the
    caller but that's not the case here. On neutron server side this
    RPC method is only calling provisioning_complete() method to
    finish provisioning of ports. And is not returning anything.

    So to make sending dhcp ready ports to neutron-server much faster
    this patch switch to use cast() method from rpc client.
    This method don't block to wait for return value from RPC caller.

    Change-Id: Ie119693854aa283b863a1eac2bdae3330c2b6a9d
    Closes-Bug: #1850864
    (cherry picked from commit 1a686fb401eca1843b81292fc88c13a2e6fd274d)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.6 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron queens-eol release.