quota_details is broken for CountableResource provided by plugins other than the core plugin

[Numan Siddique]

The neutron tempest API test - neutron.tests.tempest.api.admin.test_quotas.QuotasTest.test_detail_quotas calls the API - ""GET /v2.0/quotas/{tenant_id}/details" which is failing with the below logs in the neutron server

INFO neutron.pecan_wsgi.hooks.translation [None req-64308681-f568-4dea-961b-5c9de579ac7e admin admin] GET failed (client error): The resource could not be found.
INFO neutron.wsgi [None req-64308681-f568-4dea-961b-5c9de579ac7e admin admin] 10.0.0.7 "GET /v2.0/quotas/ff5c5121117348df94aa181d3504375b/detail HTTP/1.1" status: 404 len: 309 time: 0.0295429
ERROR neutron.api.v2.resource [None req-b1b677cd-73b1-435d-bcc4-845dfa713046 admin admin] details failed: No details.: AttributeError: 'Ml2Plugin' object has no attribute 'get_floatingips'
ERROR neutron.api.v2.resource Traceback (most recent call last):
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 98, in resource
ERROR neutron.api.v2.resource result = method(request=request, **args)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/extensions/quotasv2_detail.py", line 56, in details
ERROR neutron.api.v2.resource self._get_detailed_quotas(request, id)}
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/extensions/quotasv2_detail.py", line 46, in _get_detailed_quotas
ERROR neutron.api.v2.resource resource_registry.get_all_resources(), tenant_id)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 163, in wrapped
ERROR neutron.api.v2.resource return method(*args, **kwargs)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 93, in wrapped
ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
ERROR neutron.api.v2.resource self.force_reraise()
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 89, in wrapped
ERROR neutron.api.v2.resource return f(*args, **kwargs)
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 150, in wrapper
ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
ERROR neutron.api.v2.resource self.force_reraise()
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 138, in wrapper
ERROR neutron.api.v2.resource return f(*args, **kwargs)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 128, in wrapped
ERROR neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e)
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
ERROR neutron.api.v2.resource self.force_reraise()
ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 124, in wrapped
ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/quota/driver.py", line 98, in get_detailed_tenant_quotas
ERROR neutron.api.v2.resource used = resource.count(context, plugin, tenant_id)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/quota/resource.py", line 132, in count
ERROR neutron.api.v2.resource return self._count_func(context, plugin, self.plural_name, tenant_id)
ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/quota/resource.py", line 38, in _count_resource
ERROR neutron.api.v2.resource obj_getter = getattr(plugin, "get_%s" % collection_name)
ERROR neutron.api.v2.resource AttributeError: 'Ml2Plugin' object has no attribute 'get_floatingips'
ERROR neutron.api.v2.resource
INFO neutron.wsgi [None req-b1b677cd-73b1-435d-bcc4-845dfa713046 admin admin] 10.0.0.7 "GET /v2.0/quotas/ff5c5121117348df94aa181d3504375b/details HTTP/1.1" status: 500 len: 368 time: 0.1003060

*****

After some investigation I think the suspect is this - https://github.com/openstack/neutron/blob/master/neutron/db/quota/driver.py#L97

The commit https://github.com/openstack/neutron/commit/a8109af65f275ec1b2e725695bf3bb9976f22ae3 recently added quota_details extension.

The function "directory.get_plugins()" returns the below dict in my setup.

plugins = {'FLAVORS': <weakproxy at 0x83fb9f0 to FlavorsPlugin at 0x11e48ad0>, 'CORE': <weakproxy at 0x11f24c00 to Ml2Plugin at 0x5fa8310>, 'network-ip-availability': <weakproxy at 0x11e75ba8 to NetworkIPAvailabilityPlugin at 0x11e48a90>, 'timestamp': <weakproxy at 0x11e75c00 to TimeStampPlugin at 0x11ca5a90>, 'auto-allocated-topology': <weakproxy at 0x11e75c58 to Plugin at 0x11e4f850>, 'revision_plugin': <weakproxy at 0x11e75cb0 to RevisionPlugin at 0x11e59cd0>, 'TAG': <weakproxy at 0x11e75d08 to TagPlugin at 0x11e56550>, 'L3_ROUTER_NAT': <weakproxy at 0x11e75d60 to OVNL3RouterPlugin at 0x11acf6d0>, 'LOADBALANCERV2': <weakproxy at 0x11e75db8 to LoadBalancerPluginv2 at 0x11ad6110>}

when the code calls, plugin = plugins.get(key, plugins[constants.CORE]), the value of 'key' is floatingip.

[Numan Siddique (numansiddique)]

This could actually be a neutron issue. But I want to debug further before setting the "affects" flag.

[Numan Siddique (numansiddique)]

The issue is not seen with neutron ml2ovs plugin because, floatingip is a tracked resource.

I think for countable resources, this is a problem. Suppose loadbalancer plugin is loaded, then we will see the same error.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/500367

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/500443

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500367
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=dc11f5cedbd2b51f75c944e16a91bd94e053f53a
Submitter: Jenkins
Branch: master

commit dc11f5cedbd2b51f75c944e16a91bd94e053f53a
Author: Numan Siddique <email address hidden>
Date: Sun Sep 3 19:20:19 2017 +0530

    Track router and floatingip quota usage using TrackedResource

    Presently these resources are created as CountableResource. The
    newly added quota_details extension is not handling the countable
    resources properly. See the bug description for more details.
    It's any way better to create as trackable resources. Please see
    [1] for more details on trackable resources.

    [1] - https://docs.openstack.org/neutron/latest/contributor/internals/quota.html
    Closes-bug: #1714769
    Change-Id: I50dcb05d3d58ee6c23e59861d76da3c3ef83022b

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/501410

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/501593

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501410
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=07bfe6adb96ee0a88b9dd54d7e4b0bb684b63e3c
Submitter: Jenkins
Branch: master

commit 07bfe6adb96ee0a88b9dd54d7e4b0bb684b63e3c
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Sep 6 12:55:24 2017 -0700

    CountableResource: try count/get functions for all plugins

    It's of no guarantee that core plugin implements counter/getter function
    for a CountableResource. Instead of just trying core plugin, try every
    plugin registered in the directory.

    To retain backwards compatibility, we also make sure that core plugin is
    checked first.

    Change-Id: I5245e217e1f44281f85febbdfaf873321253dc5d
    Closes-Bug: #1714769

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/503062

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503062
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8d1b5bda3820a76d2af9758fc985e6e5e6004b33
Submitter: Jenkins
Branch: stable/pike

commit 8d1b5bda3820a76d2af9758fc985e6e5e6004b33
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Sep 6 12:55:24 2017 -0700

    CountableResource: try count/get functions for all plugins

    It's of no guarantee that core plugin implements counter/getter function
    for a CountableResource. Instead of just trying core plugin, try every
    plugin registered in the directory.

    To retain backwards compatibility, we also make sure that core plugin is
    checked first.

    Change-Id: I5245e217e1f44281f85febbdfaf873321253dc5d
    Closes-Bug: #1714769
    (cherry picked from commit 07bfe6adb96ee0a88b9dd54d7e4b0bb684b63e3c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500428
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=0217af742f74a25a1ae2c6b34dbb1f4eeb6271e1
Submitter: Jenkins
Branch: master

commit 0217af742f74a25a1ae2c6b34dbb1f4eeb6271e1
Author: YAMAMOTO Takashi <email address hidden>
Date: Mon Sep 4 12:22:38 2017 +0900

    devstack: Update NETWORK_API_EXTENSIONS

    Enable some of extensions provided by ML2 because we don't
    have our monolithic plugin anymore. Also, enable some
    new plugin agnostic extensions.

    Closes-Bug: #1714863
    Related-Bug: #1714769
    Change-Id: I58f96127d9479e845d1b28bf0986d43e2411269f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501593
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=a887f499c4c6ac4635cf0f575e77afb41fee0405
Submitter: Jenkins
Branch: master

commit a887f499c4c6ac4635cf0f575e77afb41fee0405
Author: YAMAMOTO Takashi <email address hidden>
Date: Thu Sep 7 16:13:09 2017 +0900

    devstack: Enable quota_details in NETWORK_API_EXTENSIONS

    Related-Bug: #1714769
    Depends-On: I5245e217e1f44281f85febbdfaf873321253dc5d
    Depends-On: I9bc85c9bab1fba1ae54259cde6f34f6b8d43b4a9
    Change-Id: Ic9044c8ec48950d465211cb91e4dcf26ffabd5c3

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 4.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/500443
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/500443
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/500443
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.