Creating a trunk with many subports is really slow
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
networking-ovn |
Fix Released
|
Undecided
|
Maciej Jozefczyk |
Bug Description
When running rally create and list trunk scenario [0], the create_trunk operation with 125 subports takes almost 180 seconds to complete. The reason why this happens is that we're updating the binding information for every subport [1]
If we could instead bulk update all the subports with a single call to the plugin, the operation would be much faster. On the other hand but this is not yet supported in ML2 plugin AFAIK.
[0] https:/
[1] https:/
Nate Johnston (nate-johnston) wrote : | #1 |
Daniel Alvarez (dalvarezs) wrote : | #2 |
@Nate: _set_sub_ports is called from the OVN Trunk driver [0] upon trunk creation or when adding subports to an existing trunk. The trunk driver will then call the ML2 plugin to update each subport one by one. Let me know if it doesn't make sense, please.
[0] https:/
Daniel Alvarez (dalvarezs) wrote : | #3 |
Talking to Slaweq looks like one -ugly- possibility could be to update the DB directly from networking-ovn trunk driver to avoid calling port_update on the plugin which seems to be *really* slow.
Changed in networking-ovn: | |
assignee: | nobody → Maciej Jozefczyk (maciej.jozefczyk) |
Changed in networking-ovn: | |
status: | New → In Progress |
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (master) | #4 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 41f6d622abc65ca
Author: Maciej Józefczyk <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 19.7 for 95%ile
from 99 sec to 14.9 for 50%ile
The same thing has been done for Trunk deletion.
Closes-Bug: #1834637
Change-Id: I020ba64618b2eb
Changed in networking-ovn: | |
status: | In Progress → Fix Released |
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to networking-ovn (stable/stein) | #5 |
Related fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/stein) | #6 |
Fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/rocky) | #7 |
Fix proposed to branch: stable/rocky
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/queens) | #8 |
Fix proposed to branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to networking-ovn (stable/stein) | #9 |
Related fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/stein) | #10 |
Change abandoned by Daniel Alvarez (<email address hidden>) on branch: stable/stein
Review: https:/
Reason: in favor of https:/
OpenStack Infra (hudson-openstack) wrote : Related fix merged to networking-ovn (stable/stein) | #11 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/stein
commit 9b71675a0ed1df0
Author: Daniel Alvarez <email address hidden>
Date: Tue Sep 17 11:52:21 2019 +0200
use trunk constants from neutron-lib
The trunk constants live in neutron-lib now. This patch switches over
to them in prep for https:/
Related-Bug: #1834637
Conflicts:
Change-Id: Ia5739ca39998a8
(cherry-picked from commit 814774a45791fd6
tags: | added: in-stable-stein |
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/stein) | #12 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/stein
commit 1d58b0e680ee3da
Author: Maciej Józefczyk <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 19.7 for 95%ile
from 99 sec to 14.9 for 50%ile
The same thing has been done for Trunk deletion.
Closes-Bug: #1834637
Conflicts:
Change-Id: I020ba64618b2eb
(cherry picked from commit 41f6d622abc65ca
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn 7.0.0.0b1 | #13 |
This issue was fixed in the openstack/
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/rocky) | #14 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/rocky
commit 1e2e424f5d90be3
Author: Maciej Józefczyk <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 19.7 for 95%ile
from 99 sec to 14.9 for 50%ile
The same thing has been done for Trunk deletion.
Closes-Bug: #1834637
Conflicts:
Change-Id: I020ba64618b2eb
(cherry picked from commit 843200b9ce710d6
tags: | added: in-stable-rocky |
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to networking-ovn (master) | #15 |
Related fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to networking-ovn (stable/train) | #16 |
Related fix proposed to branch: stable/train
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (master) | #17 |
Change abandoned by Jakub Libosvar (<email address hidden>) on branch: master
Review: https:/
Reason: This was supposed to go to stable/train
OpenStack Infra (hudson-openstack) wrote : Related fix merged to networking-ovn (master) | #18 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 2e0832f7b8bfc31
Author: Maciej Józefczyk <email address hidden>
Date: Fri Sep 27 06:46:40 2019 +0000
Revert "Set binding profile directly from OVNTrunkDriver"
During recent days we many observed failures of
neutron_
It started to fail on a different asserts.
Lets revert this patch, as it is a regression, and work further on this issue.
This reverts commit 41f6d622abc65ca
Related-Bug: #1834637
Related-Bug: #1845479
Change-Id: I63acf0c809aa15
OpenStack Infra (hudson-openstack) wrote : Related fix merged to networking-ovn (stable/train) | #19 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/train
commit 60ca3c7809cba35
Author: Jakub Libosvar <email address hidden>
Date: Fri Sep 27 12:23:09 2019 +0200
Revert "Set binding profile directly from OVNTrunkDriver"
During recent days we many observed failures of
neutron_
It started to fail on a different asserts.
Lets revert this patch, as it is a regression, and work further on this issue.
This reverts commit 41f6d622abc65ca
Related-Bug: #1834637
Related-Bug: #1845479
Change-Id: I244b937e916474
tags: | added: in-stable-train |
OpenStack Infra (hudson-openstack) wrote : Related fix merged to networking-ovn (stable/stein) | #20 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/stein
commit 4f2301a060ea8ce
Author: Maciej Józefczyk <email address hidden>
Date: Fri Sep 27 06:50:19 2019 +0000
Revert "Set binding profile directly from OVNTrunkDriver"
During recent days we many observed failures of
neutron_
It started to fail on a different asserts.
Lets revert this patch, as it is a regression, and work further on this issue.
This reverts commit 1d58b0e680ee3da
Related-Bug: #1834637
Related-Bug: #1845479
Change-Id: I55d224faead40d
OpenStack Infra (hudson-openstack) wrote : Related fix merged to networking-ovn (stable/rocky) | #21 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/rocky
commit 3205f887fcc9a50
Author: Maciej Józefczyk <email address hidden>
Date: Fri Sep 27 06:47:10 2019 +0000
Revert "Set binding profile directly from OVNTrunkDriver"
During recent days we many observed failures of
neutron_
It started to fail on a different asserts.
Lets revert this patch, as it is a regression, and work further on this issue.
This reverts commit 1e2e424f5d90be3
Related-Bug: #1834637
Related-Bug: #1845479
Change-Id: I862751fabcd590
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (master) | #22 |
Fix proposed to branch: master
Review: https:/
Flavio Fernandes (ffernand) wrote : | #23 |
- loop run for exercising test over the weekend Edit (3.3 MiB, text/plain)
I had these loops running over the weekend [1] and was unable to reproduce the bug after applying the 686571 changes in patchset 2.
As you can see in the output below [2], I did see some failures in test, but they were all related to nova unable to ssh into a vm [3] after creation; which I consider outside the scope of bug 1834637.
[1]:
# main shell session
while : ; do \
cd /opt/stack/tempest && while [ $? -eq 0 ]; \
do sleep 1 ; echo '' ; date +"%D %T %Z" | tee -a /vagrant/
tempest run --regex \
neutron_
done ; \
sleep 30 ; \
done
# on another session
while : ; do openstack port list ; sleep 1 ; done
# on another session
while : ; do openstack port create --network private blablabla ; \
sleep 3 ; openstack port delete blablabla && echo deleted ; sleep 3 ; done
[2]:
$ ls -lah tempest.log
-rw-r--r-- 1 flaviof flaviof 3.4M Oct 7 04:59 tempest.log
$ grep -c "... ok" tempest.log
2340
$ grep -c "... FAILED" tempest.log
27
[3]:
$ grep -c 'File "tempest/
54 <== 2 failed attempts for each one of the 27 failed tests
See attached log for more details.
Flavio Fernandes (ffernand) wrote : | #24 |
- log that shows issue before patchet 2 is applied Edit (428.0 KiB, text/plain)
Attaching tempest.
instantiated as part of transaction.
$ grep -c "... ok" tempest.
84
$ grep -c "... FAILED" tempest.
2
$ grep -nA12 "... FAILED" tempest.
472:{0} neutron_
473-
474-Captured traceback:
475-~~~
476- Traceback (most recent call last):
477- File "/opt/stack/
478- self.wait_
479- File "/opt/stack/
480- server, constants.
481- File "/opt/stack/
482- waiters.
483- File "tempest/
484- server_
--
778:{0} neutron_
779-
780-Captured traceback:
781-~~~
782- Traceback (most recent call last):
783- File "/opt/stack/
784- self._wait_
785- File "/opt/stack/
786- "status {!r}.".
787- File "/opt/stack/
788- raise exception
789- RuntimeError: Timed out waiting for port u'40509745-
790-
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (master) | #25 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 7bdf2eb824083b6
Author: Flavio Fernandes <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver (redo)
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile
The same thing has been done for Trunk deletion.
This reverts commit 2e0832f7b8bfc31
Change-Id: I6b659cbede25f2
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <email address hidden>
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/train) | #26 |
Fix proposed to branch: stable/train
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/stein) | #27 |
Fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/rocky) | #28 |
Fix proposed to branch: stable/rocky
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/queens) | #29 |
Fix proposed to branch: stable/queens
Review: https:/
tags: | added: networking-ovn-proactive-backport-potential |
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/train) | #30 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/train
commit 1e20b9bf866e300
Author: Flavio Fernandes <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver (redo)
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile
The same thing has been done for Trunk deletion.
This reverts commit 2e0832f7b8bfc31
---
Also, merge squashed:
Enable ovs-vswitchd and ovsdb-server for rally
The ovs-vswitchd and ovsdb-server services were missing in the rally
definition. Usually the jobs inherts from our base class which enables
them but rally has a different job parent.
Change-Id: I6b659cbede25f2
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <email address hidden>
Co-authored-by: Lucas Alvares Gomes <email address hidden>
(cherry picked from commit 7bdf2eb824083b6
(cherry picked from commit 202e804c64c9c6d
(cherry picked from commit 7e725331dbaf584
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/queens) | #31 |
Change abandoned by Lucas Alvares Gomes (<email address hidden>) on branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/stein) | #32 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/stein
commit c6dee38266b662a
Author: Flavio Fernandes <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver (redo)
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile
The same thing has been done for Trunk deletion.
This reverts commit 2e0832f7b8bfc31
Change-Id: I6b659cbede25f2
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit 1e20b9bf866e300
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/queens) | #33 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/queens
commit 01d86c3c97b1962
Author: Flavio Fernandes <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver (redo)
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile
The same thing has been done for Trunk deletion.
This reverts commit 2e0832f7b8bfc31
Note: While this is a back-merge from Rocky, special care was
needed to account for the fact that in Queens release there
is only one port binding for a db_port. Thus no iteration over
db_
Change-Id: I6b659cbede25f2
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit 62eb828186f9248
tags: | added: in-stable-queens |
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/rocky) | #34 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/rocky
commit 62eb828186f9248
Author: Flavio Fernandes <email address hidden>
Date: Thu Sep 5 09:29:45 2019 +0000
Set binding profile directly from OVNTrunkDriver (redo)
Setting binding profile for Trunk subports takes
time - for 125 subports rally CreateAndListTrunks
scenario [0] takes about 150 seconds. We need to
bump up the perfomance because large number of
subports is widly used in Kuryr deployments.
To achieve that I changed setting the binding
profile to be saved directly to the neutron DB.
Instead calling port_update I update only related
fields in OVN NorthBound DB rows. That gave performance
improvement in trunk port creation:
from 101 sec to 35.6 for 95%ile
from 99 sec to 34.2 for 50%ile
The same thing has been done for Trunk deletion.
This reverts commit 2e0832f7b8bfc31
Change-Id: I6b659cbede25f2
Closes-Bug: #1834637
Related-Bug: #1845479
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit c6dee38266b662a
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn 4.0.4 | #35 |
This issue was fixed in the openstack/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/queens) | #36 |
Fix proposed to branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/rocky) | #37 |
Fix proposed to branch: stable/rocky
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/stein) | #38 |
Fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/train) | #39 |
Fix proposed to branch: stable/train
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (master) | #40 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/queens) | #41 |
Fix proposed to branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (master) | #42 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (master) | #43 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (master) | #44 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/queens) | #45 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #46 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/train) | #47 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: stable/train
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/stein) | #48 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (master) | #49 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/rocky) | #50 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: stable/rocky
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/queens) | #51 |
Fix proposed to branch: stable/queens
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (master) | #52 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/train) | #53 |
Fix proposed to branch: stable/train
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/stein) | #54 |
Fix proposed to branch: stable/stein
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/rocky) | #55 |
Fix proposed to branch: stable/rocky
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (master) | #56 |
Change abandoned by Flavio Fernandes (<email address hidden>) on branch: master
Review: https:/
Reason: This is now: https:/
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/train) | #57 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/train
commit c418dd720b9be90
Author: Flavio Fernandes <email address hidden>
Date: Fri Nov 22 15:11:48 2019 -0500
Set binding profile directly from OVNTrunkDriver (redo cont.)
This is a tweak to changes done to fix bug 1834637. Specifically,
this change addresses scaling. The previous gerrit change had
modifications to all OVN sub-ports performed as a single
transaction. That did not account for race-condition on neutron
DB queries, which leads to timeouts under heavy loads.
Another cleanup done by this change is to fold an additional
update on neutron db into ovn trunk driver. That saves
update from doing another database transaction, thus making
it faster. The no longer needed function in mech_driver was
called _update_
By breaking the iteration into multiple transactions, the
change in time is marginal:
Service-level agreement
NeutronTrunks :: neutron.
from 34.2 sec to 35.6 for 50%ile
from 35.6 sec to 36.1 for 95%ile
This patch doesn't go to master as it's frozen now because of
the code being merged to Neutron. The patch will be sent to master
Neutron once code migration is finished.
Change-Id: I7de3ac2a7cf886
Closes-Bug: #1834637
Co-authored-by: Maciej Józefczyk <email address hidden>
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/stein) | #58 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/stein
commit e669382b251b5fe
Author: Flavio Fernandes <email address hidden>
Date: Fri Nov 22 15:11:48 2019 -0500
Set binding profile directly from OVNTrunkDriver (redo cont.)
This is a tweak to changes done to fix bug 1834637. Specifically,
this change addresses scaling. The previous gerrit change had
modifications to all OVN sub-ports performed as a single
transaction. That did not account for race-condition on neutron
DB queries, which leads to timeouts under heavy loads.
Another cleanup done by this change is to fold an additional
update on neutron db into ovn trunk driver. That saves
update from doing another database transaction, thus making
it faster. The no longer needed function in mech_driver was
called _update_
By breaking the iteration into multiple transactions, the
change in time is marginal:
Service-level agreement
NeutronTrunks :: neutron.
from 34.2 sec to 35.6 for 50%ile
from 35.6 sec to 36.1 for 95%ile
This patch doesn't go to master networking-ovn. It
has been migrated to master Neutron [1].
[1]: https:/
Change-Id: I7de3ac2a7cf886
Closes-Bug: #1834637
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit c418dd720b9be90
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/rocky) | #59 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/rocky
commit 7f03767fd2e1eb6
Author: Flavio Fernandes <email address hidden>
Date: Fri Nov 22 15:11:48 2019 -0500
Set binding profile directly from OVNTrunkDriver (redo cont.)
This is a tweak to changes done to fix bug 1834637. Specifically,
this change addresses scaling. The previous gerrit change had
modifications to all OVN sub-ports performed as a single
transaction. That did not account for race-condition on neutron
DB queries, which leads to timeouts under heavy loads.
Another cleanup done by this change is to fold an additional
update on neutron db into ovn trunk driver. That saves
update from doing another database transaction, thus making
it faster. The no longer needed function in mech_driver was
called _update_
By breaking the iteration into multiple transactions, the
change in time is marginal:
Service-level agreement
NeutronTrunks :: neutron.
from 34.2 sec to 35.6 for 50%ile
from 35.6 sec to 36.1 for 95%ile
This patch doesn't go to master networking-ovn. It
has been migrated to master Neutron [1].
[1]: https:/
Change-Id: I7de3ac2a7cf886
Closes-Bug: #1834637
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit c418dd720b9be90
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/queens) | #60 |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: stable/queens
commit 4bed17daa762f97
Author: Flavio Fernandes <email address hidden>
Date: Fri Nov 1 16:33:12 2019 -0400
Set binding profile directly from OVNTrunkDriver (redo cont.)
This is a tweak to changes done to fix bug 1834637. Specifically,
this change addresses scaling. The previous gerrit change had
modifications to all OVN sub-ports performed as a single
transaction. That did not account for race-condition on neutron
DB queries, which leads to timeouts under heavy loads.
Another cleanup done by this change is to fold an additional
update on neutron db into ovn trunk driver. That saves
update from doing another database transaction, thus making
it faster. The no longer needed function in mech_driver was
called _update_
By breaking the iteration into multiple transactions, the
change in time is marginal:
Service-level agreement
NeutronTrunks :: neutron.
from 34.2 sec to 35.6 for 50%ile
from 35.6 sec to 36.1 for 95%ile
This patch doesn't go to master networking-ovn. It
has been migrated to master Neutron [1].
[1]: https:/
Depends-On: https:/
Change-Id: I7de3ac2a7cf886
Closes-Bug: #1834637
Co-authored-by: Maciej Józefczyk <email address hidden>
(cherry picked from commit c418dd720b9be90
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn 7.1.0 | #61 |
This issue was fixed in the openstack/
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn 6.0.1 | #63 |
This issue was fixed in the openstack/
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 16.0.0.0b1 | #66 |
This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn 5.1.0 | #67 |
This issue was fixed in the openstack/
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn queens-eol | #70 |
This issue was fixed in the openstack/
Is _set_sub_ports being called once with information about all the subports, or is it being called from the ML2 level once per subport?