nicira: tempest fails to delete routers in parallel

Bug #1258150 reported by Salvatore Orlando
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Salvatore Orlando

Bug Description

when parallel tempest tests are enabled, the nicira plugin shows erros when deleting routers.
Parallel operations indeed cause the usual eventlet/mysql deadlock in delete_router as the nvp operation is nested within the db transaction.

The root cause for the deadlock is that the nvp api client uses eventlet to dispatch requests.

while a solution might be to rework the API client, an easier, backportable solution would be to move the NVP operation out of the transaction and ensuring consistency in case of failure.

note: in the same delete_router routine also the metada access network handling should be moved out of the transaction.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/61604

Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/61604
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2aaec3a81148e6c0cefd8f5b989dbcecf872a680
Submitter: Jenkins
Branch: master

commit 2aaec3a81148e6c0cefd8f5b989dbcecf872a680
Author: Salvatore Orlando <email address hidden>
Date: Wed Dec 11 15:10:11 2013 -0800

    NVP plugin: Do backend router delete out from db transaction

    Performing the NVP API operation from within a DB transaction
    increases the risk of a deadlock between sqlalchemy and eventlet.

    With this patch, the operation is moved outside of the db transaction
    and appropriate mechanism are put in place for:
    i) ensuring neutron db consistency in case of NVP failures
    ii) avoiding deleting from backend if neutron logic does not allow it

    This patch also synchronizes the routine for removing a router
    gateway port from NVP.

    Change-Id: I58d156e303e7a56ceb8c62766c192e154b0a3bb4
    Closes-Bug: #1258150

Changed in neutron:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in neutron:
milestone: icehouse-2 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.