2022-09-16 00:07:22.883000+00:00 - neutron-rpc-server - 2022-09-16 00:07:22,883 22 ERROR oslo_messaging.rpc.server neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler-2333834 failed with "Unknown quota resources ['security_group_rule']." 2022-09-19 00:05:00.935000+00:00 - neutron-rpc-server - 2022-09-19 00:05:00,935 31 ERROR oslo_messaging.rpc.server [req-7c64b81c-0bd4-4049-853a-fdb19a262b37 gNone - - - - -] Exception during message handling: neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources ['security_group_rule']." 2022-09-19 00:05:00.935000+00:00 - neutron-rpc-server - 2022-09-19 00:05:00,935 31 ERROR oslo_messaging.rpc.server neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources ['security_group_rule']." 2022-09-19 00:05:01.023000+00:00 - neutron-dhcp-agent - 2022-09-19 00:05:01,023 7 ERROR neutron.agent.dhcp.agent [req-7c64b81c-0bd4-4049-853a-fdb19a262b37 gNone - - - - -] Unable to enable dhcp for 6e7a8a5b-6055-4d54-a636-f96b7a6496ce.: oslo_messaging.rpc.client.RemoteError: Remote error: CallbackFailure Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources ['security_group_rule']." 2022-09-19 00:05:01.023000+00:00 - neutron-dhcp-agent - 2022-09-19 00:05:01,023 7 ERROR neutron.agent.dhcp.agent oslo_messaging.rpc.client.RemoteError: Remote error: CallbackFailure Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources ['security_group_rule']." 2022-09-19 00:05:01.023000+00:00 - neutron-dhcp-agent - 2022-09-19 00:05:01,023 7 ERROR neutron.agent.dhcp.agent ['Traceback (most recent call last): ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch result = func(ctxt, **new_args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/quota/resource_registry.py", line 94, in wrapper ret_val = f(_self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 279, in create_dhcp_port return self._port_action(plugin, context, port, \'create_port\') ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 104, in _port_action return p_utils.create_port(plugin, context, port) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/plugins/utils.py", line 336, in create_port context, {port_apidef.RESOURCE_NAME: port_data}) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner return f(self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped return method(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1402, in create_port self._before_create_port(context, port) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1364, in _before_create_port context=context, port=attrs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/registry.py", line 56, in notify _get_callback_manager().notify(resource, event, trigger, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 108, in _wrapped raise db_exc.RetryRequest(e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 103, in _wrapped return function(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 174, in notify raise exceptions.CallbackFailure(errors=errors) ', 'neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources [\'security_group_rule\']." ']. 2022-09-19 00:05:01.735180+00:00 - neutron-dhcp-agent - ['Traceback (most recent call last): ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch result = func(ctxt, **new_args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/quota/resource_registry.py", line 94, in wrapper ret_val = f(_self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 279, in create_dhcp_port return self._port_action(plugin, context, port, \'create_port\') ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 104, in _port_action return p_utils.create_port(plugin, context, port) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/plugins/utils.py", line 336, in create_port context, {port_apidef.RESOURCE_NAME: port_data}) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner return f(self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped return method(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1402, in create_port self._before_create_port(context, port) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1364, in _before_create_port context=context, port=attrs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/registry.py", line 56, in notify _get_callback_manager().notify(resource, event, trigger, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 108, in _wrapped raise db_exc.RetryRequest(e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 103, in _wrapped return function(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 174, in notify raise exceptions.CallbackFailure(errors=errors) ', 'neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources [\'security_group_rule\']." ']. 2022-09-19 00:05:02.522000+00:00 - neutron-dhcp-agent - 2022-09-19 00:05:02,522 7 DEBUG neutron.agent.dhcp.agent [req-419d7dcb-cbe7-43d0-bf74-10cb6f45f12d gNone - - - - -] resync (6e7a8a5b-6055-4d54-a636-f96b7a6496ce): [RemoteError('Remote error: CallbackFailure Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources [\'security_group_rule\']." [\'Traceback (most recent call last):\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\ res = self.dispatcher.dispatch(message)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch\ return self._do_dispatch(endpoint, method, ctxt, args)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch\ result = func(ctxt, **new_args)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\ return func(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\ setattr(e, \\\'_RETRY_EXCEEDED\\\', True)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\ return f(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\ ectxt.value = e.inner_exc\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\ return f(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\ LOG.debug("Retry wrapper got retriable exception: %s", e)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\ return f(*dup_args, **dup_kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/quota/resource_registry.py", line 94, in wrapper\ ret_val = f(_self, context, *args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 279, in create_dhcp_port\ return self._port_action(plugin, context, port, \\\'create_port\\\')\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 104, in _port_action\ return p_utils.create_port(plugin, context, port)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/plugins/utils.py", line 336, in create_port\ context, {port_apidef.RESOURCE_NAME: port_data})\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner\ return f(self, context, *args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped\ return method(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\ setattr(e, \\\'_RETRY_EXCEEDED\\\', True)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\ return f(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\ ectxt.value = e.inner_exc\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\ return f(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\ LOG.debug("Retry wrapper got retriable exception: %s", e)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\ return f(*dup_args, **dup_kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1402, in create_port\ self._before_create_port(context, port)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1364, in _before_create_port\ context=context, port=attrs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/registry.py", line 56, in notify\ _get_callback_manager().notify(resource, event, trigger, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 108, in _wrapped\ raise db_exc.RetryRequest(e)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\ self.force_reraise()\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\ six.reraise(self.type_, self.value, self.tb)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise\ raise value\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 103, in _wrapped\ return function(*args, **kwargs)\ \', \' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 174, in notify\ raise exceptions.CallbackFailure(errors=errors)\ \', \'neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036788749943 failed with "Unknown quota resources [\\\'security_group_rule\\\']."\ \'].',)] _periodic_resync_helper /var/lib/openstack/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py:356 2022-10-07 09:30:53.157000+00:00 - neutron-rpc-server - 2022-10-07 09:30:53,157 33 ERROR oslo_messaging.rpc.server [req-39528002-ee7f-4367-823a-ff316b067fae gNone - - - - -] Exception during message handling: neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources ['security_group_rule']." 2022-10-07 09:30:53.157000+00:00 - neutron-rpc-server - 2022-10-07 09:30:53,157 33 ERROR oslo_messaging.rpc.server neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources ['security_group_rule']." 2022-10-07 09:30:53.321000+00:00 - neutron-linuxbridge-agent - 2022-10-07 09:30:53,321 7 ERROR neutron.plugins.ml2.drivers.agent._common_agent [req-39528002-ee7f-4367-823a-ff316b067fae gNone - - - - -] Unable to get port details for {'tapc2499c21-11', 'tapdc0987c0-db', 'tap376911c7-37', 'tape8c6c779-2f', 'tap282fd414-0d', 'tapa712bed0-e9', 'tap57f8929d-c1', 'tap75be4579-2e', 'tap970c7cf1-42', 'tapcf4e21a6-87', 'tap4b4e9b81-29', 'tapf129ddbd-c9', 'tapc7bc7545-df', 'tap340cbf6f-c7', 'tap24fe033e-d4', 'tap188c8b85-f7', 'tap4d00e832-c9', 'tap55b3916e-e3', 'tapabbb8d42-4b', 'tapda907257-7b', 'tapb349bbcf-14', 'tapc7f131a7-11', 'tapa57cc24e-67', 'tape37dee37-1c', 'tap7895a660-53', 'tap931a8338-0f', 'tap61b15822-be', 'tap533cbdfc-54', 'tap12c5641b-db', 'tap4df614dd-4b', 'tapd7c6db46-9c', 'tapcd4839dd-62', 'tap0948bca9-13', 'tap0b6426fd-f4', 'tape350dab8-52', 'tapb858adee-0b', 'tap8a8b5676-19', 'tap542e98a4-88', 'tap517ab24f-07', 'tap12087164-bb', 'tapea2da95b-f8', 'tapc939360f-2d', 'tap504c4a1b-18', 'tap3f014112-0a', 'tap612f9127-65', 'tapac7056c5-d7', 'tapd1df4a0f-5d', 'tap2c7af853-4b', 'tap5ae1332d-40', 'tapaf56e741-20', 'tapb78d48c7-6b', 'tap9871bc24-e2', 'tape809e556-d7', 'tapb2c96959-bb'}: oslo_messaging.rpc.client.RemoteError: Remote error: CallbackFailure Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources ['security_group_rule']." 2022-10-07 09:30:53.321000+00:00 - neutron-linuxbridge-agent - 2022-10-07 09:30:53,321 7 ERROR neutron.plugins.ml2.drivers.agent._common_agent oslo_messaging.rpc.client.RemoteError: Remote error: CallbackFailure Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources ['security_group_rule']." 2022-10-07 09:30:53.321000+00:00 - neutron-linuxbridge-agent - 2022-10-07 09:30:53,321 7 ERROR neutron.plugins.ml2.drivers.agent._common_agent ['Traceback (most recent call last): ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch result = func(ctxt, **new_args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 181, in get_devices_details_list for device in kwargs.pop(\'devices\', []) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 181, in for device in kwargs.pop(\'devices\', []) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 121, in get_device_details port_context.network.current) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2066, in update_port_status context, {port_id: status}, host)[port_id] ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner return f(self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped return method(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2081, in update_port_statuses context, port_dbs_by_id[port_id], status, host) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2093, in _safe_update_individual_port_db_status ectx.reraise = bool(db.get_port(context, port_id)) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2089, in _safe_update_individual_port_db_status context, port, status, host) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2109, in _update_individual_port_db_status context=context, port=attr) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/registry.py", line 56, in notify _get_callback_manager().notify(resource, event, trigger, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 108, in _wrapped raise db_exc.RetryRequest(e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 103, in _wrapped return function(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 174, in notify raise exceptions.CallbackFailure(errors=errors) ', 'neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources [\'security_group_rule\']." ']. 2022-10-07 09:30:53.508521+00:00 - neutron-linuxbridge-agent - ['Traceback (most recent call last): ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch result = func(ctxt, **new_args) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 181, in get_devices_details_list for device in kwargs.pop(\'devices\', []) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 181, in for device in kwargs.pop(\'devices\', []) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/rpc.py", line 121, in get_device_details port_context.network.current) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2066, in update_port_status context, {port_id: status}, host)[port_id] ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner return f(self, context, *args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped return method(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped setattr(e, \'_RETRY_EXCEEDED\', True) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped LOG.debug("Retry wrapper got retriable exception: %s", e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped return f(*dup_args, **dup_kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2081, in update_port_statuses context, port_dbs_by_id[port_id], status, host) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2093, in _safe_update_individual_port_db_status ectx.reraise = bool(db.get_port(context, port_id)) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2089, in _safe_update_individual_port_db_status context, port, status, host) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 2109, in _update_individual_port_db_status context=context, port=attr) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/registry.py", line 56, in notify _get_callback_manager().notify(resource, event, trigger, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 108, in _wrapped raise db_exc.RetryRequest(e) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() ', ' File "/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/six.py", line 703, in reraise raise value ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/db/utils.py", line 103, in _wrapped return function(*args, **kwargs) ', ' File "/var/lib/openstack/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 174, in notify raise exceptions.CallbackFailure(errors=errors) ', 'neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036852234615 failed with "Unknown quota resources [\'security_group_rule\']." '].