SQL query explosion for bgpvpn

Bug #1632758 reported by Pierre Crégut
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-bgpvpn
Fix Released
High
Pierre Crégut

Bug Description

When networking-bgpbpn fetches a bgpvpn object from the database through sqlalchemy, it will use the relations with networks and routers eagerly. This means it will build a query whose skeleton is:
SELECT <columns (240) FROM bgpvpns
LEFT OUTER JOIN bgpvpn_network_associations ON bgpvpns.id = bgpvpn_network_associations.bgpvpn_id LEFT OUTER JOIN networks ON networks.id = bgpvpn_network_associations.network_id
...
LEFT OUTER JOIN bgpvpn_router_associations ON bgpvpns.id = bgpvpn_router_associations.bgpvpn_id
LEFT OUTER JOIN routers ON routers1.id = bgpvpn_router_associations.router_id
...

There are two problems:
- A huge query fetching 240 columns at once and performing 45 outer joins
- An explosion of the number of rows returned: if there are i networks and j routers, there will be i*j rows leading to a potential combinatorial explosion in the size of the result.
The problem is exacerbated by the fact that a single network or routers can be represented by several rows for the same reason (if it has several tags, subnets, rbac rules, prefix in subnetpools, explicit routes etc. )
The solution is to use lazy="select" for the relations between bgpvpn and bgpvpn_network_associations on one hand and bgpvpn_router_assocations on the other hand.

Revision history for this message
Pierre Crégut (pcregut) wrote :
Pierre Crégut (pcregut)
Changed in bgpvpn:
assignee: nobody → Pierre Crégut (pcregut)
Changed in bgpvpn:
status: New → Confirmed
importance: Undecided → High
milestone: none → 6.0.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-bgpvpn (master)

Fix proposed to branch: master
Review: https://review.openstack.org/386608

Changed in bgpvpn:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bgpvpn (master)

Reviewed: https://review.openstack.org/386608
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=88791f54670766c9a4c7009934bf01cc59ea4db5
Submitter: Jenkins
Branch: master

commit 88791f54670766c9a4c7009934bf01cc59ea4db5
Author: Pierre Crégut <email address hidden>
Date: Fri Oct 14 11:37:45 2016 +0200

    Modify bgpvpn relations with association tables to select

    Prevents networking-bgpvpn from building a large outer join that
    would lead to a cartesian product of the rows for networks and
    the rows for routers in the answers.

    Change-Id: I322635c130dc818741319ae96850f362e74c4d2f
    Closes-Bug: 1632758

Changed in bgpvpn:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-bgpvpn 6.0.0

This issue was fixed in the openstack/networking-bgpvpn 6.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.