[ovn] Memory consumption grows over time due to MAC_Binding entries in SB database

Bug #1946318 reported by Daniel Alvarez
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Daniel Alvarez

Bug Description

MAC_Binding entries are used in OVN as a mechanism to learn MAC addresses on logical ports and avoid sending ARP requests to the network.

There is no aging mechanism for these entries [0] and the table can grow indefinitely. In environments with for example large (eg. /16) external networks; OVN may learn a considerable amount of addresses growing the size of the db a lot.

Today, Neutron monitors this table to workaround the lack of aging mechanism and remove the MAC_Binding entries associated to Floating IPs and each neutron-server worker will keep an in-memory copy of such table increasing its memory footprint to several Gigabytes, eventually leading to OOM killers.

[0] https://mail.openvswitch.org/pipermail/ovs-discuss/2019-June/048936.html

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/812805

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → High
assignee: nobody → Daniel Alvarez (dalvarezs)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/813610

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/812805
Committed: https://opendev.org/openstack/neutron/commit/f6c35527698119ee6f73a6a3613c9beebb563840
Submitter: "Zuul (22348)"
Branch: master

commit f6c35527698119ee6f73a6a3613c9beebb563840
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Oct 7 09:23:21 2021 +0200

    [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint

    The MAC_Binding table in the SB database may grow indefinitely (due
    to a lack of an aging mechanism of its entries) and eventually
    lead to OOM killers for neutron-server which maintains an in-memory
    copy of the database.

    In order to stop monitoring this table, this patch is invoking
    the ovsdb-client tool to remove the entries associated to
    Floating IPs that have just been detached. The execution of this
    tool is really fast as it will just invoke a JSON-RPC transact command
    which doesn't require downloading the database contents.

    In a scale test, the memory consumption of neutron-server dropped
    from 75GB to 7GB with this patch.

    Closes-Bug: #1946318

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/813610
Committed: https://opendev.org/openstack/neutron/commit/a278c5ba789c014ec777a75fc9538179d6707202
Submitter: "Zuul (22348)"
Branch: master

commit a278c5ba789c014ec777a75fc9538179d6707202
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 12 11:42:39 2021 +0100

    [OVN] Tune OVN routers to reduce the mem footprint for ML2/OVN

    In order to avoid having a MAC_Binding table explosion and helping
    lowering the memory footprint when using ML2/OVN this patch is setting
    two options to the OVN routers:

    * always_learn_from_arp_request: By setting this to False we
      avoid learning from ARP replies observed in the network. Only the
      ARP requests sent by OVN will generate a MAC_Binding entry in the
      OVSDB database. For larger broadcasts domains this avoids having a
      MAC_Binding table explosion, reduce the DB size and memory footprint
      of ML2/OVN.

    * dynamic_neigh_routers: By setting this to True we avoid
      pre-populating flows for router to router communication, reduding
      the number of flows, DB size and memory footprint of ML2/OVN.

    For more information on these option for core OVN please refer to:
    https://www.ovn.org/support/dist-docs/ovn-nb.5.html

    This patch also includes a new maintenance task to include these options
    to existing routers in the system.

    Related-Bug: #1946318
    Change-Id: I056acdec9b6ee2341d2bc4f7bd9a678f3bf91972
    Signed-off-by: Lucas Alvares Gomes <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/814419

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/814420

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/814421

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/xena)

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/814422

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814422
Committed: https://opendev.org/openstack/neutron/commit/987f0689ead8cdb76a813de81429c159c705984f
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 987f0689ead8cdb76a813de81429c159c705984f
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 12 11:42:39 2021 +0100

    [OVN] Tune OVN routers to reduce the mem footprint for ML2/OVN

    In order to avoid having a MAC_Binding table explosion and helping
    lowering the memory footprint when using ML2/OVN this patch is setting
    two options to the OVN routers:

    * always_learn_from_arp_request: By setting this to False we
      avoid learning from ARP replies observed in the network. Only the
      ARP requests sent by OVN will generate a MAC_Binding entry in the
      OVSDB database. For larger broadcasts domains this avoids having a
      MAC_Binding table explosion, reduce the DB size and memory footprint
      of ML2/OVN.

    * dynamic_neigh_routers: By setting this to True we avoid
      pre-populating flows for router to router communication, reduding
      the number of flows, DB size and memory footprint of ML2/OVN.

    For more information on these option for core OVN please refer to:
    https://www.ovn.org/support/dist-docs/ovn-nb.5.html

    This patch also includes a new maintenance task to include these options
    to existing routers in the system.

    Related-Bug: #1946318
    Change-Id: I056acdec9b6ee2341d2bc4f7bd9a678f3bf91972
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit a278c5ba789c014ec777a75fc9538179d6707202)

tags: added: in-stable-xena
tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814421
Committed: https://opendev.org/openstack/neutron/commit/9365bbd19ff57b3636fc2f3f003f3b9531f40afc
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 9365bbd19ff57b3636fc2f3f003f3b9531f40afc
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 12 11:42:39 2021 +0100

    [OVN] Tune OVN routers to reduce the mem footprint for ML2/OVN

    In order to avoid having a MAC_Binding table explosion and helping
    lowering the memory footprint when using ML2/OVN this patch is setting
    two options to the OVN routers:

    * always_learn_from_arp_request: By setting this to False we
      avoid learning from ARP replies observed in the network. Only the
      ARP requests sent by OVN will generate a MAC_Binding entry in the
      OVSDB database. For larger broadcasts domains this avoids having a
      MAC_Binding table explosion, reduce the DB size and memory footprint
      of ML2/OVN.

    * dynamic_neigh_routers: By setting this to True we avoid
      pre-populating flows for router to router communication, reduding
      the number of flows, DB size and memory footprint of ML2/OVN.

    For more information on these option for core OVN please refer to:
    https://www.ovn.org/support/dist-docs/ovn-nb.5.html

    This patch also includes a new maintenance task to include these options
    to existing routers in the system.

    Related-Bug: #1946318
    Change-Id: I056acdec9b6ee2341d2bc4f7bd9a678f3bf91972
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit a278c5ba789c014ec777a75fc9538179d6707202)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814420
Committed: https://opendev.org/openstack/neutron/commit/34c3ec244e12e74c4fb3517d75303ec8a30fc581
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 34c3ec244e12e74c4fb3517d75303ec8a30fc581
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 12 11:42:39 2021 +0100

    [OVN] Tune OVN routers to reduce the mem footprint for ML2/OVN

    In order to avoid having a MAC_Binding table explosion and helping
    lowering the memory footprint when using ML2/OVN this patch is setting
    two options to the OVN routers:

    * always_learn_from_arp_request: By setting this to False we
      avoid learning from ARP replies observed in the network. Only the
      ARP requests sent by OVN will generate a MAC_Binding entry in the
      OVSDB database. For larger broadcasts domains this avoids having a
      MAC_Binding table explosion, reduce the DB size and memory footprint
      of ML2/OVN.

    * dynamic_neigh_routers: By setting this to True we avoid
      pre-populating flows for router to router communication, reduding
      the number of flows, DB size and memory footprint of ML2/OVN.

    For more information on these option for core OVN please refer to:
    https://www.ovn.org/support/dist-docs/ovn-nb.5.html

    This patch also includes a new maintenance task to include these options
    to existing routers in the system.

    Related-Bug: #1946318
    Change-Id: I056acdec9b6ee2341d2bc4f7bd9a678f3bf91972
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit a278c5ba789c014ec777a75fc9538179d6707202)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814419
Committed: https://opendev.org/openstack/neutron/commit/e814e93e8ed92683c5b46083fe479cf384b81ed5
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit e814e93e8ed92683c5b46083fe479cf384b81ed5
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 12 11:42:39 2021 +0100

    [OVN] Tune OVN routers to reduce the mem footprint for ML2/OVN

    In order to avoid having a MAC_Binding table explosion and helping
    lowering the memory footprint when using ML2/OVN this patch is setting
    two options to the OVN routers:

    * always_learn_from_arp_request: By setting this to False we
      avoid learning from ARP replies observed in the network. Only the
      ARP requests sent by OVN will generate a MAC_Binding entry in the
      OVSDB database. For larger broadcasts domains this avoids having a
      MAC_Binding table explosion, reduce the DB size and memory footprint
      of ML2/OVN.

    * dynamic_neigh_routers: By setting this to True we avoid
      pre-populating flows for router to router communication, reduding
      the number of flows, DB size and memory footprint of ML2/OVN.

    For more information on these option for core OVN please refer to:
    https://www.ovn.org/support/dist-docs/ovn-nb.5.html

    This patch also includes a new maintenance task to include these options
    to existing routers in the system.

    Related-Bug: #1946318
    Change-Id: I056acdec9b6ee2341d2bc4f7bd9a678f3bf91972
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit a278c5ba789c014ec777a75fc9538179d6707202)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/814869

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/814870

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/814871

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/814872

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814872
Committed: https://opendev.org/openstack/neutron/commit/5c9c366376324bc98de23709f578c454243a21a9
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 5c9c366376324bc98de23709f578c454243a21a9
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Oct 7 09:23:21 2021 +0200

    [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint

    The MAC_Binding table in the SB database may grow indefinitely (due
    to a lack of an aging mechanism of its entries) and eventually
    lead to OOM killers for neutron-server which maintains an in-memory
    copy of the database.

    In order to stop monitoring this table, this patch is invoking
    the ovsdb-client tool to remove the entries associated to
    Floating IPs that have just been detached. The execution of this
    tool is really fast as it will just invoke a JSON-RPC transact command
    which doesn't require downloading the database contents.

    In a scale test, the memory consumption of neutron-server dropped
    from 75GB to 7GB with this patch.

    Closes-Bug: #1946318

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3
    (cherry picked from commit f6c35527698119ee6f73a6a3613c9beebb563840)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814871
Committed: https://opendev.org/openstack/neutron/commit/1a9db2168a87d79947c0f1afdb431d603a0db60e
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 1a9db2168a87d79947c0f1afdb431d603a0db60e
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Oct 7 09:23:21 2021 +0200

    [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint

    The MAC_Binding table in the SB database may grow indefinitely (due
    to a lack of an aging mechanism of its entries) and eventually
    lead to OOM killers for neutron-server which maintains an in-memory
    copy of the database.

    In order to stop monitoring this table, this patch is invoking
    the ovsdb-client tool to remove the entries associated to
    Floating IPs that have just been detached. The execution of this
    tool is really fast as it will just invoke a JSON-RPC transact command
    which doesn't require downloading the database contents.

    In a scale test, the memory consumption of neutron-server dropped
    from 75GB to 7GB with this patch.

    Closes-Bug: #1946318

    Conflicts:
      neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py
      neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3
    (cherry picked from commit f6c35527698119ee6f73a6a3613c9beebb563840)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814870
Committed: https://opendev.org/openstack/neutron/commit/3ac446155225ac98d2253fdb82ff38fd7fa340a1
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 3ac446155225ac98d2253fdb82ff38fd7fa340a1
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Oct 7 09:23:21 2021 +0200

    [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint

    The MAC_Binding table in the SB database may grow indefinitely (due
    to a lack of an aging mechanism of its entries) and eventually
    lead to OOM killers for neutron-server which maintains an in-memory
    copy of the database.

    In order to stop monitoring this table, this patch is invoking
    the ovsdb-client tool to remove the entries associated to
    Floating IPs that have just been detached. The execution of this
    tool is really fast as it will just invoke a JSON-RPC transact command
    which doesn't require downloading the database contents.

    In a scale test, the memory consumption of neutron-server dropped
    from 75GB to 7GB with this patch.

    Closes-Bug: #1946318

    Conflicts:
      neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py
      neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py
      neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py

    Note: The functional test has been changed for this stable branch
    because the update_tables() method from ovsdbapp is not availabie for
    this version of Neutron. So I changed the functional test to add the
    MAC_Binding entry via ovsdb-client instead of using ovsdbapp to do so.

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3
    (cherry picked from commit f6c35527698119ee6f73a6a3613c9beebb563840)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814869
Committed: https://opendev.org/openstack/neutron/commit/b767825d1329dbb7fcff264d6047d2513d02db4e
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit b767825d1329dbb7fcff264d6047d2513d02db4e
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Oct 7 09:23:21 2021 +0200

    [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint

    The MAC_Binding table in the SB database may grow indefinitely (due
    to a lack of an aging mechanism of its entries) and eventually
    lead to OOM killers for neutron-server which maintains an in-memory
    copy of the database.

    In order to stop monitoring this table, this patch is invoking
    the ovsdb-client tool to remove the entries associated to
    Floating IPs that have just been detached. The execution of this
    tool is really fast as it will just invoke a JSON-RPC transact command
    which doesn't require downloading the database contents.

    In a scale test, the memory consumption of neutron-server dropped
    from 75GB to 7GB with this patch.

    Closes-Bug: #1946318

    Conflicts:
      neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py
      neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py
      neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py

    Note: The functional test has been changed for this stable branch
    because the update_tables() method from ovsdbapp is not availabie for
    this version of Neutron. So I changed the functional test to add the
    MAC_Binding entry via ovsdb-client instead of using ovsdbapp to do so.

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3
    (cherry picked from commit f6c35527698119ee6f73a6a3613c9beebb563840)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/816698

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 16.4.2

This issue was fixed in the openstack/neutron 16.4.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/816698
Committed: https://opendev.org/openstack/neutron/commit/7874c576013928c036dca4d9c0a38e5b8ae06bb4
Submitter: "Zuul (22348)"
Branch: master

commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command

    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.

    This patch is adding a timeout parameter to the command line to
    prevent this condition.

    Closes-Bug: #1948891
    Related-Bug: #1946318

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3

tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/xena)

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/818796

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/818799

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/818800

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/818801

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/818796
Committed: https://opendev.org/openstack/neutron/commit/6b270bd6f86784e16c9d79339d6726b74a75ca70
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 6b270bd6f86784e16c9d79339d6726b74a75ca70
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command

    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.

    This patch is adding a timeout parameter to the command line to
    prevent this condition.

    Closes-Bug: #1948891
    Related-Bug: #1946318

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3
    (cherry picked from commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/818800
Committed: https://opendev.org/openstack/neutron/commit/37333d3788dc0ab3ba510d9c655b9e8d1fc1a7ff
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 37333d3788dc0ab3ba510d9c655b9e8d1fc1a7ff
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command

    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.

    This patch is adding a timeout parameter to the command line to
    prevent this condition.

    Closes-Bug: #1948891
    Related-Bug: #1946318

    Conflicts:
      neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3
    (cherry picked from commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/818799
Committed: https://opendev.org/openstack/neutron/commit/f37e0be349f3492672badd43af05b5952610c40d
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit f37e0be349f3492672badd43af05b5952610c40d
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command

    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.

    This patch is adding a timeout parameter to the command line to
    prevent this condition.

    Closes-Bug: #1948891
    Related-Bug: #1946318

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3
    (cherry picked from commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/818801
Committed: https://opendev.org/openstack/neutron/commit/15e2da82c27f64b0fabcd6b4d8b33be9da769ac9
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 15e2da82c27f64b0fabcd6b4d8b33be9da769ac9
Author: Daniel Alvarez Sanchez <email address hidden>
Date: Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command

    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.

    This patch is adding a timeout parameter to the command line to
    prevent this condition.

    Closes-Bug: #1948891
    Related-Bug: #1946318

    Conflicts:
        neutron/tests/unit/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py

    Signed-off-by: Daniel Alvarez Sanchez <email address hidden>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3
    (cherry picked from commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.1.0

This issue was fixed in the openstack/neutron 19.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.3.0

This issue was fixed in the openstack/neutron 17.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.2.0

This issue was fixed in the openstack/neutron 18.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.0.0.0rc1

This issue was fixed in the openstack/neutron 20.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn train-eol

This issue was fixed in the openstack/networking-ovn train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.