ODL and Neutron can get out of sync due to various conditions

[John Joyce]

The design of the ODL ML2 driver on the southbound side of neutron combined with the design of the Northbound side of ODL interface to neutron is subject to various race conditions that can cause a loss of synchronization.

These race conditions can be triggered in various scenarios. Some of these scenarios have been shared via: https://wiki.opendaylight.org/images/8/8d/Experiences_with_Neutron.pdf

Some cases include:

When the driver attempts to synchronize the two databases especially a full sync other threads may be changing the database and causing corruption.
In a cluster HA scenarios strict ordering of the transactions called out to ODL can't be guaranteed to be in the same order as entered in the neutron database during the pre-commit phase.

It is likely that the scope of the changes necessary to eliminate these race conditions maybe pretty large.

A similar bug:
https://bugs.launchpad.net/networking-odl/+bug/1371115
was previously entered, but is more limited in scope to a particular case. This bug being broader in scope should ensure that the specific scenario noted is also addressed.

[Isaku Yamahata (yamahata)]

Similar bug
https://bugs.launchpad.net/networking-odl/+bug/1489553

[Isaku Yamahata (yamahata)]

Since the description of this is better than 1489553, 1489553 is marked as incomplete and let's keep this bug as unified one.

[OpenStack Infra (hudson-openstack)]

Change abandoned by rcurran (<email address hidden>) on branch: master
Review: https://review.openstack.org/253722
Reason: Moved UT to https://review.openstack.org/#/c/222409/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/222409
Committed: https://git.openstack.org/cgit/openstack/networking-odl/commit/?id=78f656d95cf031772a315e6d9b1c95e57eaf9a8a
Submitter: Jenkins
Branch: master

commit 78f656d95cf031772a315e6d9b1c95e57eaf9a8a
Author: Arvind Somya <email address hidden>
Date: Mon Aug 31 12:23:58 2015 -0400

    Opendaylight driver refactor to handle out of sync issues

    This commits refactors the Opendaylight ML2 mechanism driver to
    handle race conditions and out of sync issues. It attempts to
    address these using an atomic thread synchronization model that
    tracks the sync, retry, success and failed states of Neutron objects
    sent to Opendaylight.

    Partial-Bug #1493551

    Co-Authored-By: Arvind Somya (<email address hidden>)
    Co-Authored-By: Rich Curran (<email address hidden>)

    Change-Id: I654321a9012aba98775690d530e3e0788957b21b

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/266583
Committed: https://git.openstack.org/cgit/openstack/networking-odl/commit/?id=177d68fb85f0b6b748c7938f391c9c5f560fbddc
Submitter: Jenkins
Branch: master

commit 177d68fb85f0b6b748c7938f391c9c5f560fbddc
Author: Rich Curran <email address hidden>
Date: Tue Jan 12 16:43:33 2016 -0500

    Opendaylight L3 service plugin refactor to handle out of sync issues

    This commits refactors the Opendaylight L3 service plugin to handle race
    conditions and out of sync issues. It attempts to address these using an
    atomic thread synchronization model that tracks the sync, retry, success
    and failed states of Neutron objects sent to Opendaylight.

    Partial-Bug: #1493551

    Change-Id: I8664e05a5294a1ba3e455a9b31ea984621c02f8a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/281490
Committed: https://git.openstack.org/cgit/openstack/networking-odl/commit/?id=8ec4c21449b3189f06bb934a2d5a78f4cfe140ff
Submitter: Jenkins
Branch: master

commit 8ec4c21449b3189f06bb934a2d5a78f4cfe140ff
Author: Rich Curran <email address hidden>
Date: Wed Feb 17 14:24:55 2016 -0500

    ODL v2: Security Group support

    This commits refactors the Opendaylight security group code to handle
    race conditions and out of sync issues. It attempts to address these
    using an atomic thread synchronization model that tracks the sync,
    retry, success and failed states of Neutron objects sent to Opendaylight.

    Change-Id: Ia1742bfd30aeef0ad9216e4b02d6a97751b276c3
    Partial-Bug: #1493551

[Isaku Yamahata (yamahata)]

As this bug report is quite old, and this issue is being addressed as v2 driver effort.
Close this report.