ObectDeletedError while deleting network

Bug #1454423 reported by Eugene Nikanorov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Oleg Bondarev

Bug Description

Bug in upstream: https://bugs.launchpad.net/neutron/+bug/1454408

The following trace could be observed running rally tests on multi-server environment:

2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource Traceback (most recent call last):
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 87, in resource
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource result = method(request=request, **args)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 476, in delete
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource obj_deleter(request.context, id, **kwargs)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 671, in delete_network
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource self._delete_ports(context, ports)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 587, in _delete_ports
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource port.id)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 239, in __get__
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource return self.impl.get(instance_state(instance), dict_)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 589, in get
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource value = callable_(state, passive)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/state.py", line 424, in __call__
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource self.manager.deferred_scalar_loader(self, toload)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/loading.py", line 614, in load_scalar_attributes
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource raise orm_exc.ObjectDeletedError(state)
2015-05-12 11:41:20.503 14172 TRACE neutron.api.v2.resource ObjectDeletedError: Instance '<Port at 0x7fb50067e290>' has been deleted, or its row is otherwise not present.

Tags: neutron
tags: added: neutron
Changed in mos:
milestone: none → 7.0
Changed in mos:
status: Confirmed → Incomplete
Revision history for this message
Oleg Bondarev (obondarev) wrote :

Need to backport fix from upstream https://review.openstack.org/#/c/182456/

Changed in mos:
status: Incomplete → Confirmed
importance: Medium → High
description: updated
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/neutron (openstack-ci/fuel-7.0/2015.1.0)

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Eugene Nikanorov <email address hidden>
Review: https://review.fuel-infra.org/10001

Changed in mos:
assignee: Eugene Nikanorov (enikanorov) → Oleg Bondarev (obondarev)
status: Confirmed → In Progress
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/neutron (openstack-ci/fuel-7.0/2015.1.0)

Reviewed: https://review.fuel-infra.org/10001
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: af3b8af99d5ff6de21ecbaf773ee2303081ed7ef
Author: Eugene Nikanorov <email address hidden>
Date: Fri Jul 31 09:23:28 2015

Catch ObjectDeletedError and skip port or subnet removal

When network is deleted service ports are deleted in the scope of
delete_network. Service ports could also be deleted by other entities
such as DHCP agent releasing dhcp port.
That could rarely lead to a race condition when port object used in
_delete_ports helper is already deleted causing ObjectDeletedError
exception.
Need to handle it and prevent object deletion in that case.

Closes-Bug: #1454408
Closes-Bug: #1454423

Change-Id: I531251d3211545c82a5bb7a471b7915da9b763b7

Changed in mos:
status: In Progress → Fix Committed
oleksii shyman (oshyman)
tags: added: on-verification
oleksii shyman (oshyman)
Changed in mos:
status: Fix Committed → Fix Released
tags: removed: on-verification
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.