Race condition when creating neutron_pg_drop port group

[Daniel Alvarez]

The neutron_pg_drop Port Group is created with some ACLs to drop all the traffic by default.
This Port Group doesn't match any Neutron resource and it's created once the first Neutron port with port_security enabled gets created.

A race condition exists when the first two ports get created simultaneously by different workers as both may attempt to create this Port Group. The result is that one of the port creations will fail.

A patch like this should fix it:

--- a/networking_ovn/common/ovn_client.py
+++ b/networking_ovn/common/ovn_client.py
@@ -1792,11 +1792,11 @@ class OVNClient(object):
         with self._nb_idl.transaction(check_error=True) as txn:
             if not self._nb_idl.get_port_group(pg_name):
                 # If drop Port Group doesn't exist yet, create it.
- txn.add(self._nb_idl.pg_add(pg_name, acls=[]))
+ txn.add(self._nb_idl.pg_add(pg_name, may_exist=True, acls=[]))
                 # Add ACLs to this Port Group so that all traffic is dropped.
                 acls = ovn_acl.add_acls_for_drop_port_group(pg_name)
                 for acl in acls:
- txn.add(self._nb_idl.pg_acl_add(**acl))
+ txn.add(self._nb_idl.pg_acl_add(**acl, may_exist=True))

             if ports:
                 ports_ids = [port['id'] for port in ports]

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/683246

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/683246
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=e946836b208a5ddc3a17f2dba53658c4b9a3f8f7
Submitter: Zuul
Branch: master

commit e946836b208a5ddc3a17f2dba53658c4b9a3f8f7
Author: Terry Wilson <email address hidden>
Date: Thu Sep 19 16:55:17 2019 -0500

    Avoid port group creation race

    The neutron_pg_drop Porg Group is created with some ACLs to drop
    all the traffic by default. This group doesn't match any Neutron
    resource and it's created once the first Neutron port with
    port_security enabled gets created.

    A race condition exists whn the first two ports get created
    simultaneously by different works as both may attempt to create
    this Port Group. The result is that one of the port creations will
    fail. Setting may_exist=True should avoid this race

    Co-authored-by: Daniel Alvarez <email address hidden>
    Closes-Bug: 1838969
    Change-Id: I8c1174df562a7164d4a96ce2e733279723827dcd

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/687566

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/687567

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/687568

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/687569

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687566
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=4b1c92c8bf09c86763ee1244f66953b6d4193484
Submitter: Zuul
Branch: stable/train

commit 4b1c92c8bf09c86763ee1244f66953b6d4193484
Author: Terry Wilson <email address hidden>
Date: Thu Sep 19 16:55:17 2019 -0500

    Avoid port group creation race

    The neutron_pg_drop Porg Group is created with some ACLs to drop
    all the traffic by default. This group doesn't match any Neutron
    resource and it's created once the first Neutron port with
    port_security enabled gets created.

    A race condition exists whn the first two ports get created
    simultaneously by different works as both may attempt to create
    this Port Group. The result is that one of the port creations will
    fail. Setting may_exist=True should avoid this race

    Co-authored-by: Daniel Alvarez <email address hidden>
    Closes-Bug: 1838969
    Change-Id: I8c1174df562a7164d4a96ce2e733279723827dcd
    (cherry picked from commit e946836b208a5ddc3a17f2dba53658c4b9a3f8f7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687567
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=ac13fe61310765796b29d33c7df9e8103eddb147
Submitter: Zuul
Branch: stable/stein

commit ac13fe61310765796b29d33c7df9e8103eddb147
Author: Terry Wilson <email address hidden>
Date: Thu Sep 19 16:55:17 2019 -0500

    Avoid port group creation race

    The neutron_pg_drop Porg Group is created with some ACLs to drop
    all the traffic by default. This group doesn't match any Neutron
    resource and it's created once the first Neutron port with
    port_security enabled gets created.

    A race condition exists whn the first two ports get created
    simultaneously by different works as both may attempt to create
    this Port Group. The result is that one of the port creations will
    fail. Setting may_exist=True should avoid this race

    Co-authored-by: Daniel Alvarez <email address hidden>
    Closes-Bug: 1838969
    Change-Id: I8c1174df562a7164d4a96ce2e733279723827dcd
    (cherry picked from commit e946836b208a5ddc3a17f2dba53658c4b9a3f8f7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687569
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=528925bb20b622e3355411f57ba2fe8bdcffd386
Submitter: Zuul
Branch: stable/queens

commit 528925bb20b622e3355411f57ba2fe8bdcffd386
Author: Terry Wilson <email address hidden>
Date: Thu Sep 19 16:55:17 2019 -0500

    Avoid port group creation race

    The neutron_pg_drop Porg Group is created with some ACLs to drop
    all the traffic by default. This group doesn't match any Neutron
    resource and it's created once the first Neutron port with
    port_security enabled gets created.

    A race condition exists whn the first two ports get created
    simultaneously by different works as both may attempt to create
    this Port Group. The result is that one of the port creations will
    fail. Setting may_exist=True should avoid this race

    Co-authored-by: Daniel Alvarez <email address hidden>
    Closes-Bug: 1838969
    Change-Id: I8c1174df562a7164d4a96ce2e733279723827dcd
    (cherry picked from commit e946836b208a5ddc3a17f2dba53658c4b9a3f8f7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687568
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=fa0d4773b2e47cea7f52109ff0aa6f6c211e4e31
Submitter: Zuul
Branch: stable/rocky

commit fa0d4773b2e47cea7f52109ff0aa6f6c211e4e31
Author: Terry Wilson <email address hidden>
Date: Thu Sep 19 16:55:17 2019 -0500

    Avoid port group creation race

    The neutron_pg_drop Porg Group is created with some ACLs to drop
    all the traffic by default. This group doesn't match any Neutron
    resource and it's created once the first Neutron port with
    port_security enabled gets created.

    A race condition exists whn the first two ports get created
    simultaneously by different works as both may attempt to create
    this Port Group. The result is that one of the port creations will
    fail. Setting may_exist=True should avoid this race

    Co-authored-by: Daniel Alvarez <email address hidden>
    Closes-Bug: 1838969
    Change-Id: I8c1174df562a7164d4a96ce2e733279723827dcd
    (cherry picked from commit e946836b208a5ddc3a17f2dba53658c4b9a3f8f7)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 4.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 7.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 6.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 5.1.0 release.