StaleDataError in disassociate_floatingips

Bug #1513242 reported by Edgar Cantu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Expired
Medium
Unassigned

Bug Description

If a VM with a floating IP is deleted while concurrently deleting the floating IP (i.e. "nova delete" and "neutron floatingip-delete") the VM may go into error state.

Nova receives a 500 error code from neutron because neutron fails to delete the port and the following stack is observed:

2015-11-04 21:48:37.447 17275 ERROR neutron.api.v2.resource [req-764172bc-4938-440d-9e6a-dd4c0f120f1b ] delete failed
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource Traceback (most recent call last):
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 83, in resource
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource result = method(request=request, **args)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/base.py", line 490, in delete
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource obj_deleter(request.context, id, **kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 131, in wrapper
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource return f(*args, **kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line 1272, in delete_port
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context, id, do_notify=False)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 274, in disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource do_notify=do_notify)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1361, in disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context, port_id)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1088, in disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource 'router_id': None})
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 483, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.rollback()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 480, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.commit()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 385, in commit
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self._prepare_impl()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 365, in _prepare_impl
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.session.flush()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1986, in flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self._flush(objects)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2104, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource transaction.rollback(_capture_exception=True)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2068, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource flush_context.execute()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource rec.execute(self)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource uow
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 61, in save_obj
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource mapper, table, update)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File "/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 537, in _emit_update_statements
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource (table.description, len(update), rows))
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource StaleDataError: UPDATE statement on table 'floatingips' expected to update 1 row(s); 0 were matched.
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource

Version: stable/kilo

Revision history for this message
Kyle Mestery (mestery) wrote :

Thanks for the bug report! Is there a way you could try this on Liberty or master to verify it's existence there? Kilo is in security only mode at this time.

Changed in neutron:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Edgar Cantu (ecantu) wrote :

I don't have a Liberty environment at the moment. It's probably going to be another 2 weeks before I do, but I'll update the bug then with my findings.

Revision history for this message
Edgar Cantu (ecantu) wrote :

I'm unable to reproduce this problem in Liberty. We can probably close this.

Changed in neutron:
assignee: nobody → Mohammed Ashraf (mohammed-asharaf)
Changed in neutron:
status: Triaged → In Progress
Changed in neutron:
status: In Progress → Incomplete
assignee: Mohammed Ashraf (mohammed-asharaf) → nobody
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for neutron because there has been no activity for 60 days.]

Changed in neutron:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.