Qos plugin performs too many queries

[Lucian Petrut]

Whenever retrieving the port list while having the QoS plugin enabled, Neutron performs about 10 DB queries per port, most of them being QoS related: http://paste.openstack.org/raw/800461/

For 1000 ports, we end up with 10 000 sequential DB queries. A simple "neutron port-list" or "nova list" command will exceed 1 minute, which is likely to hit timeouts.

This seems to be the problem: https://github.com/openstack/neutron/blob/17.0.0/neutron/db/db_base_plugin_v2.py#L1566-L1570

For each of the retrieved ports, the plugins are then supposed to provide additional details, so for each port we get a certain number of extra queries.

One idea would be to add a flag such as 'detailed' or 'include_extensions' to 'get_ports' and then propagate it to '_make_port_dict' through the 'process_extensions' parameter. Another idea would be to let the plugins extend the query but that might be less feasible.

Worth mentioning that there were a couple of commits meant to reduce the number of queries but it's still excessive:
https://review.opendev.org/c/openstack/neutron/+/667998
https://review.opendev.org/c/openstack/neutron/+/667981/

[Oleg Bondarev (obondarev)]

Going to apply same approach as in https://review.opendev.org/c/openstack/neutron/+/763777

[Lucian Petrut (petrutlucian94)]

@Oleg: Thanks for taking care of this issue. Indeed, both plugins would benefit from a similar approach.

@Sean: I'm really sorry about filing the bug on Cinder, thanks for moving it!

[Oleg Bondarev (obondarev)]

Partial fix: https://review.opendev.org/c/openstack/neutron/+/764433/

[Oleg Bondarev (obondarev)]

Fix: https://review.opendev.org/c/openstack/neutron/+/764454/

[Oleg Bondarev (obondarev)]

@Lucian, can you please try above patches on your environment?

[Lucian Petrut (petrutlucian94)]

@Oleg from what I can tell, that "bulk" check should avoid the DB queries when listing ports. I'll give it a try ASAP.

[Lucian Petrut (petrutlucian94)]

@Oleg I finally got the time to test the patches. We were hitting those issues on Train, so I've backported the patches there.

I did have an issue: the port_res['resource_request'] fields used internally (qos_id, vnic_type and network_id) are leaked to Nova, which then rejects live migrations: http://paste.openstack.org/raw/801620/

Worth mentioning that this Nova check was removed in Wallaby and backported to Ussuri: https://review.opendev.org/c/openstack/nova/+/738482. Still, I guess it would be safer if we avoided leaking those fields.

I'm attaching an archive containing the Train backports as well as the patch that avoids leaking those resource request fields in case anyone's interested in it.

[Lucian Petrut (petrutlucian94)]

Minor correction: the Nova check was removed by this patch and wasn't backported: https://review.opendev.org/c/openstack/nova/+/735570

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Both patches are backported up to Train:
- https://review.opendev.org/q/Iad29cb34825adaa8c766d01b192a6bbe9992148b
- https://review.opendev.org/q/I1d0b3975ae6e92e34e9da20a0e26ce024422d332

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.3.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.3.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 17.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.