neutron

ipam leaks DBReference error on deleted subnet

Bug #1623402 reported by Kevin Benton on 2016-09-14

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	neutron	Fix Released	High	Kevin Benton	neutron newton-rc1 "rc1"

Bug Description

spotted in rally run with lots of concurrent subnet operations:

http://logs.openstack.org/11/369511/3/check/gate-rally-dsvm-neutron-rally/b188655/logs/screen-q-svc.txt.gz#_2016-09-14_07_20_10_547

2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource [req-14b41117-5a1b-4b0d-9962-14e9adb7048a c_rally_b76aa7ea_hhhcxI8u -] delete failed: Exception deleting fixed_ip from port 1098dc9c-dd9e-4a11-8e6e-b999218d55aa
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 79, in resource
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource result = method(request=request, **args)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 555, in delete
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return self._delete(request, id, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 577, in _delete
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource obj_deleter(request.context, id, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(self, context, *args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1169, in delete_subnet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource "port %s"), port_id)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1161, in delete_subnet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.update_port(context, port_id, data)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(self, context, *args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1383, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource port)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1221, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource new_port=new_port)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/ipam_backend_mixin.py", line 739, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource new_port.get('mac_address'))
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 369, in update_port_with_ips
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource revert_on_fail=False)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 339, in update_port_with_ips
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource ip['subnet_id'], ip['ip_address'])
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/opt/stack/new/neutron/neutron/db/db_base_plugin_common.py", line 104, in _delete_ip_allocation
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource subnet_id=subnet_id)):
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2760, in __iter__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.session._autoflush()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1293, in _autoflush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self.flush()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self._flush(objects)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource transaction.rollback(_capture_exception=True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource compat.reraise(exc_type, exc_value, exc_tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource flush_context.execute()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource rec.execute(self)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource uow
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 174, in save_obj
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource mapper, table, insert)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 767, in _emit_insert_statements
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource execute(statement, multiparams)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return meth(self, multiparams, params)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource return connection._execute_clauseelement(self, multiparams, params)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource compiled_sql, distilled_params
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource context)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource util.raise_from_cause(newraise, exc_info)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource reraise(type(exception), exception, tb=exc_tb, cause=cause)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource context)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource cursor.execute(statement, parameters)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 167, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource result = self._query(query)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 323, in _query
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource conn.query(q)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 836, in query
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1020, in _read_query_result
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource result.read()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1303, in read
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource first_packet = self.connection._read_packet()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 982, in _read_packet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource packet.check_error()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource err.raise_mysql_exception(self._data)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource _check_mysql_exception(errinfo)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource raise errorclass(errno, errorvalue)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource DBReferenceError: (pymysql.err.IntegrityError) (1452, u'Cannot add or update a child row: a foreign key constraint fails (`neutron`.`ipamallocations`, CONSTRAINT `ipamallocations_ibfk_1` FOREIGN KEY (`ipam_subnet_id`) REFERENCES `ipamsubnets` (`id`) ON DELETE CASCADE)') [SQL: u'INSERT INTO ipamallocations (ip_address, status, ipam_subnet_id) VALUES (%(ip_address)s, %(status)s, %(ipam_subnet_id)s)'] [parameters: {'status': 'ALLOCATED', 'ip_address': '1.1.0.74', 'ipam_subnet_id': u'1f01b696-8306-43d3-83e0-2642922043b9'}]
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource

Tags:

Kevin Benton (kevinbenton) on 2016-09-14

Changed in neutron:
milestone:	none → newton-rc1
assignee:	nobody → Kevin Benton (kevinbenton)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-14: Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/369992

Changed in neutron:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-14: Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/370251

Armando Migliaccio (armando-migliaccio) on 2016-09-14

Changed in neutron:
importance:	Undecided → High

Carl Baldwin (carl-baldwin) on 2016-09-15

tags:

added: l3-ipam-dhcp

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-16: Fix merged to neutron (master)

Reviewed: https://review.openstack.org/369992
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a397792ff7ac03f4f021e8d7df62d04d6641aa14
Submitter: Jenkins
Branch: master

commit a397792ff7ac03f4f021e8d7df62d04d6641aa14
Author: Kevin Benton <email address hidden>
Date: Tue Sep 13 20:23:42 2016 -0700

Catch DBReferenceError in IPAM and convert to SubnetNotFound

    If a subnet is removed after lookup but before IP allocation
    insert, we can end up getting a DBReferenceError in the IPAM
    system. Since this can occur on any flush we force a flush
    with a session context manager to get the DBReference error
    in a desired spot (in multi-writer galera this is a deadlock
    much later, which we retry). We then convert it to SubnetNotFound,
    which is an expected exception.

Closes-Bug: #1623402
Change-Id: I65632b174f8692cd465c721e285526827842a740

Changed in neutron:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-19: Change abandoned on neutron (master)

Change abandoned by Kevin Benton (<email address hidden>) on branch: master
Review: https://review.openstack.org/370251
Reason: i think the issue has been resolved enough that this isn't necessary

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-26: Fix included in openstack/neutron 9.0.0.0rc1

This issue was fixed in the openstack/neutron 9.0.0.0rc1 release candidate.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.