Race condition of post-commit ovn update
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
networking-ovn |
Fix Released
|
Medium
|
Lucas Alvares Gomes |
Bug Description
In ML2 mechanism driver, OVN updates are performed in post-commit, which is after neutron DB transaction. With multi-workers/
I had a simple test with below patch to make the problem easy to be reproduced (test port_update here).
diff --git a/networking_
index e3e5050..277f487 100644
--- a/networking_
+++ b/networking_
@@ -11,7 +11,8 @@
# License for the specific language governing permissions and limitations
# under the License.
#
-
+from time import sleep
+from random import randint
import collections
from neutron_lib.api import validators
@@ -568,6 +569,7 @@ class OVNMechanismDri
def _update_
+ time.sleep(
With this change I can easily get inconsistency between Neutron and OVN. Below is the example test for updating IPv4 address for same port at same time by 2 clients.
$ neutron port-update 182165b7-
[1] 29415
[2] 29416
$ Updated port: 182165b7-
[2]+ Done neutron port-update 182165b7-
$ Updated port: 182165b7-
[1]+ Done neutron port-update 182165b7-
$ neutron port-show 182165b7-
+------
| Field | Value |
+------
...
| fixed_ips | {"subnet_id": "d34a3b2f-cb0e- |
| | 417c-8125-
| | "10.0.0.8"} |
| | {"subnet_id": "1be4ecac- |
| | 2df5-4b18-
| | "ip_address": |
| | "fd2a:66ba:
| id | 182165b7-
...
+------
$ ovn-nbctl lsp-get-addresses 182165b7-
fa:16:3e:8d:00:65 10.0.0.10 fd2a:66ba:
We can see that in Neutron side the final result is IP "10.0.0.8", but in OVN north DB the final result is "10.0.0.10" for this port.
This race condition exists for most update operations that requires change to OVN side. It seems to be a general problem of neutron ML2 plugin (although the same problem existed on the old networking-ovn monolithic plugin).
description: | updated |
Changed in networking-ovn: | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in networking-ovn: | |
assignee: | nobody → Han Zhou (zhouhan) |
Changed in networking-ovn: | |
milestone: | none → 2015.1.1 |
milestone: | 2015.1.1 → none |
I think this could impact create and delete operations as well since those are done in postcommit.