Sachin, we have seen this issue one more time today in the same control node:
2017-10-30 10:59:34.814 27662 DEBUG neutron.api.v2.base [req-d2e3cd05-177b-47b7-bdca-2241fafe5ba0 d48614806c614c6186c8a447772a5421 dfbaf5a8ab774ee89c34cd9e2a64ef32 - - -] Request body: {u'subnet': {u'name': u'jd28148-corp-site-jd28148-corp-site-subnet0-jz3nmebr7fhz', u'enable_dhcp': True, u'network_id': u'a7f4c1a2-4443-4f86-8f5e-9a262e405bab', u'dns_nameservers': [], u'ip_version': 4, u'cidr': u'28.148.0.0/24'}} prepare_request_body /usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:662 2017-10-30 10:59:34.816 27662 INFO neutron.quota [req-d2e3cd05-177b-47b7-bdca-2241fafe5ba0 d48614806c614c6186c8a447772a5421 dfbaf5a8ab774ee89c34cd9e2a64ef32 - - -] Loaded quota_driver: <neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver object at 0x7f3015561e10>. 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource [req-d2e3cd05-177b-47b7-bdca-2241fafe5ba0 d48614806c614c6186c8a447772a5421 dfbaf5a8ab774ee89c34cd9e2a64ef32 - - -] create failed 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource Traceback (most recent call last): 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 84, in resource 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource result = method(request=request, **args) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 410, in create 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource return self._create(request, body, **kwargs) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource self.force_reraise() 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 521, in _create 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource obj = do_create(body) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 503, in do_create 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource request.context, reservation.reservation_id) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource self.force_reraise() 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 496, in do_create 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource return obj_creator(request.context, **kwargs) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron_plugin_contrail/plugins/opencontrail/contrail_plugin_base.py", line 276, in create_subnet 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource subnet_created = self._create_resource('subnet', context, subnet) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron_plugin_contrail/plugins/opencontrail/contrail_plugin.py", line 286, in _create_resource 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource obj_name=res_type) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron_plugin_contrail/plugins/opencontrail/contrail_plugin.py", line 256, in _transform_response 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource plugin_base._raise_contrail_error(info, obj_name) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron_plugin_contrail/plugins/opencontrail/contrail_plugin_base.py", line 111, in _raise_contrail_error 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource raise exc.NeutronException(**info) 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource NeutronException: An unknown exception occurred. 2017-10-30 10:59:34.893 27662 ERROR neutron.api.v2.resource
===
This issue will block COX deployment.
Sachin, we have seen this issue one more time today in the same control node:
2017-10-30 10:59:34.814 27662 DEBUG neutron.api.v2.base [req-d2e3cd05- 177b-47b7- bdca-2241fafe5b a0 d48614806c614c6 186c8a447772a54 21 dfbaf5a8ab774ee 89c34cd9e2a64ef 32 - - -] Request body: {u'subnet': {u'name': u'jd28148- corp-site- jd28148- corp-site- subnet0- jz3nmebr7fhz' , u'enable_dhcp': True, u'network_id': u'a7f4c1a2- 4443-4f86- 8f5e-9a262e405b ab', u'dns_nameservers': [], u'ip_version': 4, u'cidr': u'28.148.0.0/24'}} prepare_ request_ body /usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py: 662 177b-47b7- bdca-2241fafe5b a0 d48614806c614c6 186c8a447772a54 21 dfbaf5a8ab774ee 89c34cd9e2a64ef 32 - - -] Loaded quota_driver: <neutron_ plugin_ contrail. plugins. opencontrail. quota.driver. QuotaDriver object at 0x7f3015561e10>. api.v2. resource [req-d2e3cd05- 177b-47b7- bdca-2241fafe5b a0 d48614806c614c6 186c8a447772a54 21 dfbaf5a8ab774ee 89c34cd9e2a64ef 32 - - -] create failed api.v2. resource Traceback (most recent call last): api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ resource. py", line 84, in resource api.v2. resource result = method( request= request, **args) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py" , line 410, in create api.v2. resource return self._create( request, body, **kwargs) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_db/ api.py" , line 148, in wrapper api.v2. resource ectxt.value = e.inner_exc api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 220, in __exit__ api.v2. resource self.force_ reraise( ) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 196, in force_reraise api.v2. resource six.reraise( self.type_ , self.value, self.tb) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_db/ api.py" , line 138, in wrapper api.v2. resource return f(*args, **kwargs) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py" , line 521, in _create api.v2. resource obj = do_create(body) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py" , line 503, in do_create api.v2. resource request.context, reservation. reservation_ id) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 220, in __exit__ api.v2. resource self.force_ reraise( ) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_utils/ excutils. py", line 196, in force_reraise api.v2. resource six.reraise( self.type_ , self.value, self.tb) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py" , line 496, in do_create api.v2. resource return obj_creator( request. context, **kwargs) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron_ plugin_ contrail/ plugins/ opencontrail/ contrail_ plugin_ base.py" , line 276, in create_subnet api.v2. resource subnet_created = self._create_ resource( 'subnet' , context, subnet) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron_ plugin_ contrail/ plugins/ opencontrail/ contrail_ plugin. py", line 286, in _create_resource api.v2. resource obj_name=res_type) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron_ plugin_ contrail/ plugins/ opencontrail/ contrail_ plugin. py", line 256, in _transform_response api.v2. resource plugin_ base._raise_ contrail_ error(info, obj_name) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron_ plugin_ contrail/ plugins/ opencontrail/ contrail_ plugin_ base.py" , line 111, in _raise_ contrail_ error api.v2. resource raise exc.NeutronExce ption(* *info) api.v2. resource NeutronException: An unknown exception occurred. api.v2. resource
2017-10-30 10:59:34.816 27662 INFO neutron.quota [req-d2e3cd05-
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
2017-10-30 10:59:34.893 27662 ERROR neutron.
===
This issue will block COX deployment.