[OVN] Enable ovn-monitor-all to help with OVN scale

Bug #1936781 reported by Kamil Sambor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Undecided
Kamil Sambor

Bug Description

Description of problem:

With conditional monitoring enabled in OVN, southbound ovsdb-server takes lot of time in handling the monitoring and sending the updates to all its connected clients. Its takes lot of CPU.

With monitor-all option, all ovn-controllers do not enable conditional monitoring there by reducing the load on the Southbound ovsdb-server.

To enable monitor-all option, CMS/cloud operator should run the below command on each chassis

ovs-vsctl set open . external_ids:ovn-monitor-all=true

Version-Release number of selected component (if applicable):

Kamil Sambor (ksambor)
Changed in tripleo:
assignee: nobody → Kamil Sambor (ksambor)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)
Changed in tripleo:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/801459
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/bc934d18a47bca5dcfb78f9ab903d3a6042706f5
Submitter: "Zuul (22348)"
Branch: master

commit bc934d18a47bca5dcfb78f9ab903d3a6042706f5
Author: Kamil Sambor <email address hidden>
Date: Tue Jul 20 14:23:31 2021 +0200

    Enable ovn-monitor-all option to ovn-controller

    With conditional monitoring enabled in OVN, southbound ovsdb-serve
    takes lot of time in handling the monitoring and sending the updates
    to all its connected clients. Its takes lot of CPU. With monitor-all
    option, all ovn-controllers do not enable conditional monitoring there
    by reducing the load on the Southbound ovsdb-server.
    Enable this for all deployments is-lowrisk and user shoudn't have
    posibilities to modified this manually form tht level.

    Depends-On: https://review.opendev.org/c/openstack/puppet-ovn/+/801455
    Closes-Bug: #1936781
    Signed-off-by: Kamil Sambor <email address hidden>
    Change-Id: I9014ad0c7fae391dec4ad70bcbc0728667d413c5

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/806890

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 15.1.0

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/ussuri)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (stable/ussuri)

Change abandoned by "Kamil Sambor <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816669

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816959

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/806890
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/d15f5926244ca3bda5230b052a4dac937e1af308
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit d15f5926244ca3bda5230b052a4dac937e1af308
Author: Kamil Sambor <email address hidden>
Date: Tue Jul 20 14:23:31 2021 +0200

    Enable ovn-monitor-all option to ovn-controller

    With conditional monitoring enabled in OVN, southbound ovsdb-serve
    takes lot of time in handling the monitoring and sending the updates
    to all its connected clients. Its takes lot of CPU. With monitor-all
    option, all ovn-controllers do not enable conditional monitoring there
    by reducing the load on the Southbound ovsdb-server.
    Enable this for all deployments is-lowrisk and user shoudn't have
    posibilities to modified this manually form tht level.

    Depends-On: https://review.opendev.org/c/openstack/puppet-ovn/+/801455
    Closes-Bug: #1936781
    Signed-off-by: Kamil Sambor <email address hidden>
    Change-Id: I9014ad0c7fae391dec4ad70bcbc0728667d413c5
    (cherry picked from commit bc934d18a47bca5dcfb78f9ab903d3a6042706f5)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816959
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/ba0ef35293216ba2fc379b3b834bd183389794c9
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit ba0ef35293216ba2fc379b3b834bd183389794c9
Author: Kamil Sambor <email address hidden>
Date: Tue Jul 20 14:23:31 2021 +0200

    Enable ovn-monitor-all option to ovn-controller

    With conditional monitoring enabled in OVN, southbound ovsdb-serve
    takes lot of time in handling the monitoring and sending the updates
    to all its connected clients. Its takes lot of CPU. With monitor-all
    option, all ovn-controllers do not enable conditional monitoring there
    by reducing the load on the Southbound ovsdb-server.
    Enable this for all deployments is-lowrisk and user shoudn't have
    posibilities to modified this manually form tht level.

    Depends-On: https://review.opendev.org/c/openstack/puppet-ovn/+/816955
    Closes-Bug: #1936781
    Signed-off-by: Kamil Sambor <email address hidden>
    Change-Id: I9014ad0c7fae391dec4ad70bcbc0728667d413c5
    (cherry picked from commit bc934d18a47bca5dcfb78f9ab903d3a6042706f5)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816669
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/e6de1f0827c1ab0efb8498ad829ce0e40772e7df
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit e6de1f0827c1ab0efb8498ad829ce0e40772e7df
Author: Kamil Sambor <email address hidden>
Date: Tue Jul 20 14:23:31 2021 +0200

    Enable ovn-monitor-all option to ovn-controller

    With conditional monitoring enabled in OVN, southbound ovsdb-serve
    takes lot of time in handling the monitoring and sending the updates
    to all its connected clients. Its takes lot of CPU. With monitor-all
    option, all ovn-controllers do not enable conditional monitoring there
    by reducing the load on the Southbound ovsdb-server.
    Enable this for all deployments is-lowrisk and user shoudn't have
    posibilities to modified this manually form tht level.

    Depends-On: https://review.opendev.org/c/openstack/puppet-ovn/+/801455
    Closes-Bug: #1936781
    Signed-off-by: Kamil Sambor <email address hidden>
    Change-Id: I9014ad0c7fae391dec4ad70bcbc0728667d413c5

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/817108
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/f9331baf0aa762f1726276e5b4ea9d9494045dd7
Submitter: "Zuul (22348)"
Branch: stable/train

commit f9331baf0aa762f1726276e5b4ea9d9494045dd7
Author: Kamil Sambor <email address hidden>
Date: Tue Jul 20 14:23:31 2021 +0200

    Enable ovn-monitor-all option to ovn-controller

    With conditional monitoring enabled in OVN, southbound ovsdb-serve
    takes lot of time in handling the monitoring and sending the updates
    to all its connected clients. Its takes lot of CPU. With monitor-all
    option, all ovn-controllers do not enable conditional monitoring there
    by reducing the load on the Southbound ovsdb-server.
    Enable this for all deployments is-lowrisk and user shoudn't have
    posibilities to modified this manually form tht level.

    Depends-On: https://review.opendev.org/c/openstack/puppet-ovn/+/816958
    Closes-Bug: #1936781
    Signed-off-by: Kamil Sambor <email address hidden>
    Change-Id: I9014ad0c7fae391dec4ad70bcbc0728667d413c5

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 12.4.6

This issue was fixed in the openstack/tripleo-heat-templates 12.4.6 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 13.6.0

This issue was fixed in the openstack/tripleo-heat-templates 13.6.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates train-eol

This issue was fixed in the openstack/tripleo-heat-templates train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.