Improve "OVSFirewallDriver.process_trusted_ports"

[Rodolfo Alonso]

When "OVSFirewallDriver.process_trusted_ports" is called with many ports, "_initialize_egress_no_port_security" retrieves the VIF ports ("Interface" registers in OVS DB), one per iteration, based in the port_id. Instead of this procedure, if the DB is called only once to retrieve all the VIF ports, the performance increase is noticeable. E.g.: bridge with 1000 ports and interfaces.

port_ids = ['id%s' % i for i in range(1, 1000)]
ts1 = timeutils.utcnow_ts(microsecond=True)
vifs = ovs.get_vifs_by_ids(port_ids)
ts2 = timeutils.utcnow_ts(microsecond=True)
print("Time lapsed: %s" % str(ts2 - ts1))

ts1 = timeutils.utcnow_ts(microsecond=True)
for i in range(1, 1000):
    id = "id%s" % i
    vif = ovs.get_vif_port_by_id(id)
ts2 = timeutils.utcnow_ts(microsecond=True)
print("Time lapsed: %s" % str(ts2 - ts1))

Retrieving 100 ports:
- Bulk operation: 0.08 secs
- Loop operation: 5.6 secs

Retrieving 300 ports:
- Bulk operation: 0.08 secs
- Loop operation: 16.44 secs

Retrieving 1000 ports:
- Bulk operation: 0.08 secs
- Loop operation: 59 secs

[1]https://github.com/openstack/neutron/blob/06754907e241af76570f19301093c2abab97e627/neutron/agent/linux/openvswitch_firewall/firewall.py#L667
[2]https://github.com/openstack/neutron/blob/06754907e241af76570f19301093c2abab97e627/neutron/agent/linux/openvswitch_firewall/firewall.py#L747

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/670162

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/670162
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ae1d36fa9d8e2115a5241b5da2e941cdefa2c463
Submitter: Zuul
Branch: master

commit ae1d36fa9d8e2115a5241b5da2e941cdefa2c463
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jul 10 18:57:02 2019 +0000

    Improve "OVSFirewallDriver.process_trusted_ports"

    FirewallDriver.process_trusted_ports" is called with many ports,
    "_initialize_egress_no_port_security" retrieves the VIF ports
    ("Interface" registers in OVS DB), one per iteration, based in the
    port_id. Instead of this procedure, if the DB is called only once to
    retrieve all the VIF ports, the performance increase is noticeable.
    E.g.: bridge with 1000 ports and interfaces.

    Retrieving 100 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 5.6 secs

    Retrieving 1000 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 59 secs

    Closes-Bug: #1836095
    Related-Bug: #1836023

    Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/705186

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/705187

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/705188

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705186
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=843eccb9ee481f6eb5c47a84f91f8f8487d8a7c1
Submitter: Zuul
Branch: stable/stein

commit 843eccb9ee481f6eb5c47a84f91f8f8487d8a7c1
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jul 10 18:57:02 2019 +0000

    Improve "OVSFirewallDriver.process_trusted_ports"

    FirewallDriver.process_trusted_ports" is called with many ports,
    "_initialize_egress_no_port_security" retrieves the VIF ports
    ("Interface" registers in OVS DB), one per iteration, based in the
    port_id. Instead of this procedure, if the DB is called only once to
    retrieve all the VIF ports, the performance increase is noticeable.
    E.g.: bridge with 1000 ports and interfaces.

    Retrieving 100 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 5.6 secs

    Retrieving 1000 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 59 secs

    Closes-Bug: #1836095
    Related-Bug: #1836023

    Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
    (cherry picked from commit ae1d36fa9d8e2115a5241b5da2e941cdefa2c463)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705187
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7d10d29020f3b2d4c0331f8becd81677c61121a9
Submitter: Zuul
Branch: stable/rocky

commit 7d10d29020f3b2d4c0331f8becd81677c61121a9
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jul 10 18:57:02 2019 +0000

    Improve "OVSFirewallDriver.process_trusted_ports"

    FirewallDriver.process_trusted_ports" is called with many ports,
    "_initialize_egress_no_port_security" retrieves the VIF ports
    ("Interface" registers in OVS DB), one per iteration, based in the
    port_id. Instead of this procedure, if the DB is called only once to
    retrieve all the VIF ports, the performance increase is noticeable.
    E.g.: bridge with 1000 ports and interfaces.

    Retrieving 100 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 5.6 secs

    Retrieving 1000 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 59 secs

    Closes-Bug: #1836095
    Related-Bug: #1836023

    Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
    (cherry picked from commit ae1d36fa9d8e2115a5241b5da2e941cdefa2c463)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705188
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f26f986f1325750747f3510a6cb8f125308eba01
Submitter: Zuul
Branch: stable/queens

commit f26f986f1325750747f3510a6cb8f125308eba01
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jul 10 18:57:02 2019 +0000

    Improve "OVSFirewallDriver.process_trusted_ports"

    FirewallDriver.process_trusted_ports" is called with many ports,
    "_initialize_egress_no_port_security" retrieves the VIF ports
    ("Interface" registers in OVS DB), one per iteration, based in the
    port_id. Instead of this procedure, if the DB is called only once to
    retrieve all the VIF ports, the performance increase is noticeable.
    E.g.: bridge with 1000 ports and interfaces.

    Retrieving 100 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 5.6 secs

    Retrieving 1000 ports:
    - Bulk operation: 0.08 secs
    - Loop operation: 59 secs

    Closes-Bug: #1836095
    Related-Bug: #1836023

    Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
    (cherry picked from commit ae1d36fa9d8e2115a5241b5da2e941cdefa2c463)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.7 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron queens-eol release.