subnet listing is too slow with rbac

[Eugene Nikanorov]

subnet listing of 100 subnets takes about 2 seconds on a powerfull hardware.
60% of the time is consumed by the calculation of 'shared' attribute of the subnet which involves rbac rules.

This makes horizon barely usable as number of networks grow.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/256609

[Miguel Angel Ajo (mangelajo)]

Could this be similar to: https://bugs.launchpad.net/neutron/+bug/1513782 ?

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/257222

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/256609
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f5b950dffecc049df11981dcfb5574121be3eb69
Submitter: Jenkins
Branch: master

commit f5b950dffecc049df11981dcfb5574121be3eb69
Author: Kevin Benton <email address hidden>
Date: Fri Dec 11 09:56:01 2015 -0800

    Separate rbac calculation from _make_network_dict

    When a subnet dict was being created, it was calling
    _make_network_dict to get the 'shared' flag for the
    subnet. The issue with this is that the _make_network_dict
    function would iterate over the subnets on the passed in
    network object, which would trigger a database lookup
    of all of the subnets.

    This patch just separates the 'shared' flag calculation out
    into a separate function that both calls can leverage.

    Change-Id: I2cb766ce1fd8ddcc75209f9e92221a3b77015ea2
    Closes-Bug: #1525295
    Partial-Bug: #1513782

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/258346

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/258346
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=31f184733946b7915f1a1927076deb19e4cbe6f2
Submitter: Jenkins
Branch: stable/liberty

commit 31f184733946b7915f1a1927076deb19e4cbe6f2
Author: Kevin Benton <email address hidden>
Date: Fri Dec 11 09:56:01 2015 -0800

    Separate rbac calculation from _make_network_dict

    When a subnet dict was being created, it was calling
    _make_network_dict to get the 'shared' flag for the
    subnet. The issue with this is that the _make_network_dict
    function would iterate over the subnets on the passed in
    network object, which would trigger a database lookup
    of all of the subnets.

    This patch just separates the 'shared' flag calculation out
    into a separate function that both calls can leverage.

    Change-Id: I2cb766ce1fd8ddcc75209f9e92221a3b77015ea2
    Closes-Bug: #1525295
    Partial-Bug: #1513782
    (cherry picked from commit f5b950dffecc049df11981dcfb5574121be3eb69)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/257222
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4885ef4885de072b0321858a2ae334b708da4bea
Submitter: Jenkins
Branch: master

commit 4885ef4885de072b0321858a2ae334b708da4bea
Author: Kevin Benton <email address hidden>
Date: Mon Dec 14 00:44:16 2015 -0800

    Add tests that constrain db query count

    This patch adds unit tests to ML2 and L3 that ensure that the
    number of DB calls during list operations for ports, networks,
    subnets, routers, and floating IPs remains constant regardless
    of the number of ports.

    These will prevent changes from slipping in that result in
    a separate DB query for each object in a list operation
    (for changes to the extensions used by ML2 and the DVR plugin).

    Related-Bug: #1525295
    Related-Bug: #1513782
    Related-Bug: #1525423
    Related-Bug: #1525740
    Related-Bug: #1526644

    Change-Id: I1958fc7c318bbf73238a3ad5be133fa7800c8290

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/liberty
Review: https://review.openstack.org/267467

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b2 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 7.0.2 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/267467
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0d5d7c72feaa0a25b799632cefcd873933d2a28b
Submitter: Jenkins
Branch: stable/liberty

commit 0d5d7c72feaa0a25b799632cefcd873933d2a28b
Author: Kevin Benton <email address hidden>
Date: Mon Dec 14 00:44:16 2015 -0800

    Add tests that constrain db query count

    This patch adds unit tests to ML2 and L3 that ensure that the
    number of DB calls during list operations for ports, networks,
    subnets, routers, and floating IPs remains constant regardless
    of the number of ports.

    These will prevent changes from slipping in that result in
    a separate DB query for each object in a list operation
    (for changes to the extensions used by ML2 and the DVR plugin).

    Related-Bug: #1525295
    Related-Bug: #1513782
    Related-Bug: #1525423
    Related-Bug: #1525740
    Related-Bug: #1526644

    Conflicts:
     doc/source/devref/effective_neutron.rst
     neutron/tests/unit/extensions/test_l3.py

    Change-Id: I1958fc7c318bbf73238a3ad5be133fa7800c8290
    (cherry picked from commit 4885ef4885de072b0321858a2ae334b708da4bea)