I have noticed a query that takes about 50s, looks like it's to retrieve networks. With a simple change this one now takes 0.5s.
For some reasons that query returns 27840 rows, but let's not focus too much on that.
```
SELECT networks.project_id AS networks_project_id, networks.id AS networks_id, networks.name AS networks_name, networks.status AS networks_status, networks.admin_state_up AS networks_admin_state_up, networks.vlan_transparent AS networks_vlan_transparent, networks.availability_zone_hints AS networks_availability_zone_hints, networks.mtu AS networks_mtu, networks.standard_attr_id AS networks_standard_attr_id, standardattributes_1.id AS standardattributes_1_id, standardattributes_1.resource_type AS standardattributes_1_resource_type, standardattributes_1.description AS standardattributes_1_description, standardattributes_1.revision_number AS standardattributes_1_revision_number, standardattributes_1.created_at AS standardattributes_1_created_at, standardattributes_1.updated_at AS standardattributes_1_updated_at, networkdnsdomains_1.network_id AS networkdnsdomains_1_network_id, networkdnsdomains_1.dns_domain AS networkdnsdomains_1_dns_domain, externalnetworks_1.network_id AS externalnetworks_1_network_id, externalnetworks_1.is_default AS externalnetworks_1_is_default, standardattributes_2.id AS standardattributes_2_id, standardattributes_2.resource_type AS standardattributes_2_resource_type, standardattributes_2.description AS standardattributes_2_description, standardattributes_2.revision_number AS standardattributes_2_revision_number, standardattributes_2.created_at AS standardattributes_2_created_at, standardattributes_2.updated_at AS standardattributes_2_updated_at, networksegments_1.id AS networksegments_1_id, networksegments_1.network_id AS networksegments_1_network_id, networksegments_1.network_type AS networksegments_1_network_type, networksegments_1.physical_network AS networksegments_1_physical_network, networksegments_1.segmentation_id AS networksegments_1_segmentation_id, networksegments_1.is_dynamic AS networksegments_1_is_dynamic, networksegments_1.segment_index AS networksegments_1_segment_index, networksegments_1.name AS networksegments_1_name, networksegments_1.standard_attr_id AS networksegments_1_standard_attr_id, segmenthostmappings_1.segment_id AS segmenthostmappings_1_segment_id, segmenthostmappings_1.host AS segmenthostmappings_1_host, networksecuritybindings_1.network_id AS networksecuritybindings_1_network_id, networksecuritybindings_1.port_security_enabled AS networksecuritybindings_1_port_security_enabled, qos_network_policy_bindings_1.policy_id AS qos_network_policy_bindings_1_policy_id, qos_network_policy_bindings_1.network_id AS qos_network_policy_bindings_1_network_id FROM networks LEFT OUTER JOIN networkrbacs ON networks.id = networkrbacs.object_id LEFT OUTER JOIN standardattributes AS standardattributes_1 ON standardattributes_1.id = networks.standard_attr_id LEFT OUTER JOIN networkdnsdomains AS networkdnsdomains_1 ON networks.id = networkdnsdomains_1.network_id LEFT OUTER JOIN externalnetworks AS externalnetworks_1 ON networks.id = externalnetworks_1.network_id LEFT OUTER JOIN networksegments AS networksegments_1 ON networks.id = networksegments_1.network_id LEFT OUTER JOIN standardattributes AS standardattributes_2 ON standardattributes_2.id = networksegments_1.standard_attr_id LEFT OUTER JOIN segmenthostmappings AS segmenthostmappings_1 ON networksegments_1.id = segmenthostmappings_1.segment_id LEFT OUTER JOIN networksecuritybindings AS networksecuritybindings_1 ON networks.id = networksecuritybindings_1.network_id LEFT OUTER JOIN qos_network_policy_bindings AS qos_network_policy_bindings_1 ON networks.id = qos_network_policy_bindings_1.network_id WHERE networkrbacs.action = 'access_as_shared' AND (networkrbacs.target_tenant = '*' OR networkrbacs.target_tenant = 'd8df5c234580454ab8ea0aeb4903b03f') ORDER BY networks.id ASC, networks.standard_attr_id ASC;
```
As usually the query is doing outer join on networks to add details regarding several others tables like segments, dns domains...
We can notice the filter on networkrbacs only:
```
WHERE networkrbacs.action = 'access_as_shared' AND (networkrbacs.target_tenant = '*' OR networkrbacs.target_tenant = 'd8df5c234580454ab8ea0aeb4903b03f') ORDER BY networks.id ASC, networks.standard_attr_id ASC;
```
```
27840 rows in set (55.454 sec)
```
The filter is only based on `networkrbacs`. There are about 200 000 rows. Besides that `networks` has about 200 rows.
We know that a network can be associated to N rbacs so if we could filter networks that match the project_id we can expect a limited number or rows to examine for rbacs.
I have updated the condition to add a filter on `networks.project_id` :
```
WHERE networks.project_id = 'd8df5c234580454ab8ea0aeb4903b03f' OR networkrbacs.action = 'access_as_shared' AND (networkrbacs.target_tenant = '*' OR networkrbacs.target_tenant = 'd8df5c234580454ab8ea0aeb4903b03f') ORDER BY networks.id ASC, networks.standard_attr_id ASC;
```
The result is now:
```
27840 rows in set (0.465 sec)
```
I understand the work that your are doing but I think the current model is quite good, it just needs to be optimized.
For that we should review the queries that takes time to identified issue and fix them. may be related to index or filter not well used.
I'm going to work on some of the queries to see how we can improve them. probably that one first.
What are your feeling guys?
@Tyler, do you think that you can do some tests in your env just to see if you get the same results?
An other interesting point.
I have noticed a query that takes about 50s, looks like it's to retrieve networks. With a simple change this one now takes 0.5s.
For some reasons that query returns 27840 rows, but let's not focus too much on that.
``` project_ id, networks.id AS networks_id, networks.name AS networks_name, networks.status AS networks_status, networks. admin_state_ up AS networks_ admin_state_ up, networks. vlan_transparen t AS networks_ vlan_transparen t, networks. availability_ zone_hints AS networks_ availability_ zone_hints, networks.mtu AS networks_mtu, networks. standard_ attr_id AS networks_ standard_ attr_id, standardattribu tes_1.id AS standardattribu tes_1_id, standardattribu tes_1.resource_ type AS standardattribu tes_1_resource_ type, standardattribu tes_1.descripti on AS standardattribu tes_1_descripti on, standardattribu tes_1.revision_ number AS standardattribu tes_1_revision_ number, standardattribu tes_1.created_ at AS standardattribu tes_1_created_ at, standardattribu tes_1.updated_ at AS standardattribu tes_1_updated_ at, networkdnsdomai ns_1.network_ id AS networkdnsdomai ns_1_network_ id, networkdnsdomai ns_1.dns_ domain AS networkdnsdomai ns_1_dns_ domain, externalnetwork s_1.network_ id AS externalnetwork s_1_network_ id, externalnetwork s_1.is_ default AS externalnetwork s_1_is_ default, standardattribu tes_2.id AS standardattribu tes_2_id, standardattribu tes_2.resource_ type AS standardattribu tes_2_resource_ type, standardattribu tes_2.descripti on AS standardattribu tes_2_descripti on, standardattribu tes_2.revision_ number AS standardattribu tes_2_revision_ number, standardattribu tes_2.created_ at AS standardattribu tes_2_created_ at, standardattribu tes_2.updated_ at AS standardattribu tes_2_updated_ at, networksegments _1.id AS networksegments _1_id, networksegments _1.network_ id AS networksegments _1_network_ id, networksegments _1.network_ type AS networksegments _1_network_ type, networksegments _1.physical_ network AS networksegments _1_physical_ network, networksegments _1.segmentation _id AS networksegments _1_segmentation _id, networksegments _1.is_dynamic AS networksegments _1_is_dynamic, networksegments _1.segment_ index AS networksegments _1_segment_ index, networksegments _1.name AS networksegments _1_name, networksegments _1.standard_ attr_id AS networksegments _1_standard_ attr_id, segmenthostmapp ings_1. segment_ id AS segmenthostmapp ings_1_ segment_ id, segmenthostmapp ings_1. host AS segmenthostmapp ings_1_ host, networksecurity bindings_ 1.network_ id AS networksecurity bindings_ 1_network_ id, networksecurity bindings_ 1.port_ security_ enabled AS networksecurity bindings_ 1_port_ security_ enabled, qos_network_ policy_ bindings_ 1.policy_ id AS qos_network_ policy_ bindings_ 1_policy_ id, qos_network_ policy_ bindings_ 1.network_ id AS qos_network_ policy_ bindings_ 1_network_ id FROM networks LEFT OUTER JOIN networkrbacs ON networks.id = networkrbacs. object_ id LEFT OUTER JOIN standardattributes AS standardattribu tes_1 ON standardattribu tes_1.id = networks. standard_ attr_id LEFT OUTER JOIN networkdnsdomains AS networkdnsdomains_1 ON networks.id = networkdnsdomai ns_1.network_ id LEFT OUTER JOIN externalnetworks AS externalnetworks_1 ON networks.id = externalnetwork s_1.network_ id LEFT OUTER JOIN networksegments AS networksegments_1 ON networks.id = networksegments _1.network_ id LEFT OUTER JOIN standardattributes AS standardattribu tes_2 ON standardattribu tes_2.id = networksegments _1.standard_ attr_id LEFT OUTER JOIN segmenthostmappings AS segmenthostmapp ings_1 ON networksegments _1.id = segmenthostmapp ings_1. segment_ id LEFT OUTER JOIN networksecurity bindings AS networksecurity bindings_ 1 ON networks.id = networksecurity bindings_ 1.network_ id LEFT OUTER JOIN qos_network_ policy_ bindings AS qos_network_ policy_ bindings_ 1 ON networks.id = qos_network_ policy_ bindings_ 1.network_ id WHERE networkrbacs.action = 'access_as_shared' AND (networkrbacs. target_ tenant = '*' OR networkrbacs. target_ tenant = 'd8df5c23458045 4ab8ea0aeb4903b 03f') ORDER BY networks.id ASC, networks. standard_ attr_id ASC;
SELECT networks.project_id AS networks_
```
As usually the query is doing outer join on networks to add details regarding several others tables like segments, dns domains...
We can notice the filter on networkrbacs only:
``` target_ tenant = '*' OR networkrbacs. target_ tenant = 'd8df5c23458045 4ab8ea0aeb4903b 03f') ORDER BY networks.id ASC, networks. standard_ attr_id ASC;
WHERE networkrbacs.action = 'access_as_shared' AND (networkrbacs.
```
```
27840 rows in set (55.454 sec)
```
The filter is only based on `networkrbacs`. There are about 200 000 rows. Besides that `networks` has about 200 rows.
We know that a network can be associated to N rbacs so if we could filter networks that match the project_id we can expect a limited number or rows to examine for rbacs.
I have updated the condition to add a filter on `networks. project_ id` :
``` 4ab8ea0aeb4903b 03f' OR networkrbacs.action = 'access_as_shared' AND (networkrbacs. target_ tenant = '*' OR networkrbacs. target_ tenant = 'd8df5c23458045 4ab8ea0aeb4903b 03f') ORDER BY networks.id ASC, networks. standard_ attr_id ASC;
WHERE networks.project_id = 'd8df5c23458045
```
The result is now:
```
27840 rows in set (0.465 sec)
```
I understand the work that your are doing but I think the current model is quite good, it just needs to be optimized.
For that we should review the queries that takes time to identified issue and fix them. may be related to index or filter not well used.
I'm going to work on some of the queries to see how we can improve them. probably that one first.
What are your feeling guys?
@Tyler, do you think that you can do some tests in your env just to see if you get the same results?