Creating FIP takes time

[Maciej Jozefczyk]

I noticed on upstream and downstream gates that while creating FloatingIP for action like:

neutron floatingip-create public

For ml2/ovs and ml2/ovn this operation takes minimum ~4 seconds.

The same we can find on u/s gates from rally jobs [1].

While we put the load on Neutron server it normally takes more than 10 seconds.

For ML/OVN creating a FIP doesn't end with creating NAT entry in OVN NBDB row. So its clearly only API operation.

Maybe we can consider profiling it?

[1] https://98a898dcf3dfb1090155-da3b599be5166de1dcb38898c60ea3c9.ssl.cf5.rackcdn.com/729588/1/check/neutron-rally-task/dd55aa7/results/report.html#/NeutronNetworks.associate_and_dissociate_floating_ips/overview

[Lajos Katona (lajos-katona)]

I think this is the profiled output for the given job:
https://98a898dcf3dfb1090155-da3b599be5166de1dcb38898c60ea3c9.ssl.cf5.rackcdn.com/729588/1/check/neutron-rally-task/dd55aa7/results/report.html#/NeutronNetworks.associate_and_dissociate_floating_ips/output
(wait a little till it is fully loaded)

There it is clearly visible wsgi things are the longest for creating floating ip, but I am not an expert on profiling so have no idea about the depths of this specific profiling for FIPs

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

I've compared (in my dev environment) the time spent *only by the server* in some common operations:
- Port creation: 0.6 secs
- Network creation: 1.0 secs
- Subnet creation: 1.1 secs
- FIP creation: 1.1 secs

FIP creation should be on par with the mentioned ones. Maybe there something in the client (a second request maybe) that slows down the request.

Regards.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/737047

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/737047
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f43f5dc64f13ac95657bf5abb96abd6fa7ce3079
Submitter: Zuul
Branch: master

commit f43f5dc64f13ac95657bf5abb96abd6fa7ce3079
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Jun 19 16:32:25 2020 +0000

    Use network.external DB model parameter when creating a floating IP

    When a floating IP is being created, the network provided should be
    external. Instead of quering the DB to find out if the
    "externalnetworks" DB register exists, the "network" register is
    retrieved and the "external" parameter is used (loaded using a
    back reference relationship). This will avoid one DB access.

    Change-Id: Iead245da166ee2ae691227bb18ae377fe0af4c04
    Related-Bug: #1880969

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/755844

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/755844
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=00f1d325bd30dd6bfd9e1dde94c5df6acda51549
Submitter: Zuul
Branch: master

commit 00f1d325bd30dd6bfd9e1dde94c5df6acda51549
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Oct 2 16:29:25 2020 +0000

    Do not retrieve again the Floating IP object during creation

    Floating IP OVO instance contains a reference to the DB object.
    During a DB transaction (inside a session context), this reference
    is always updated. There is no need to retrieve again the OVO
    to read the updated version of the DB object.

    For example, if a QoS policy is assigned to this Floating IP, the
    DB object will update the "qos_policy_binding" reference.

    Change-Id: Iec2552362f6c52842f12e20798324d4180d993e5
    Related-Bug: #1880969

[Brian Haley (brian-haley)]

I just checked a review on master and both the OVS and OVN rally jobs still take longer to create a floating IP than to create other objects, average was around 2.5sec for fip, only 1.5sec for network. So I guess these two fixes didn't completely solve the issue.

[Brian Haley (brian-haley)]

Looking at some recent logs these values seem Ok now, so will close this. If we see the issue again can open a new bug.