Activity log for bug #1906487

Date Who What changed Old value New value Message
2020-12-02 08:47:03 LIU Yulong bug added bug
2020-12-02 08:49:26 LIU Yulong description We recently tested the openvswitch-agent flow installation performance, we have following two results: 1. the AGENT_RES_PROCESSING_STEP is 100 (default) We can see the 4296 OpenFlow rules are installed in 20s. 2020-12-02 16:01:32.843 792 INFO neutron.agent.common.ovs_lib [None req-106cf38e-e20a-443e-b1ad-24ab1b7dc7cb - - - - -] Processing 4296 OpenFlow rules. 2020-12-02 16:01:52.698 792 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-106cf38e-e20a-443e-b1ad-24ab1b7dc7cb - - - - -] process_network_ports - iteration:266 - agent port security group processed in 31.044 2020-12-02T08:01:32.303Z|00967|connmgr|INFO|br-int<->unix#440: 1 flow_mods in the last 0 s (1 adds) 2020-12-02T08:01:33.353Z|00968|connmgr|INFO|br-int<->unix#443: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:33.864Z|00969|connmgr|INFO|br-int<->unix#446: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:34.435Z|00970|connmgr|INFO|br-int<->unix#449: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:34.966Z|00971|connmgr|INFO|br-int<->unix#452: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:35.399Z|00972|connmgr|INFO|br-int<->unix#455: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:35.904Z|00973|connmgr|INFO|br-int<->unix#458: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:36.335Z|00974|connmgr|INFO|br-int<->unix#461: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:36.820Z|00975|connmgr|INFO|br-int<->unix#464: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:37.288Z|00976|connmgr|INFO|br-int<->unix#467: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:37.805Z|00977|connmgr|INFO|br-int<->unix#470: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:38.329Z|00978|connmgr|INFO|br-int<->unix#473: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:38.792Z|00979|connmgr|INFO|br-int<->unix#476: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:39.248Z|00980|connmgr|INFO|br-int<->unix#479: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:39.687Z|00981|connmgr|INFO|br-int<->unix#482: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.164Z|00982|connmgr|INFO|br-int<->unix#485: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.568Z|00983|connmgr|INFO|br-int<->unix#488: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.978Z|00984|connmgr|INFO|br-int<->unix#491: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:41.453Z|00985|connmgr|INFO|br-int<->unix#494: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:41.892Z|00986|connmgr|INFO|br-int<->unix#497: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:42.264Z|00987|connmgr|INFO|br-int<->unix#500: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:42.639Z|00988|connmgr|INFO|br-int<->unix#503: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.077Z|00989|connmgr|INFO|br-int<->unix#506: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.452Z|00990|connmgr|INFO|br-int<->unix#509: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.879Z|00991|connmgr|INFO|br-int<->unix#512: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:44.385Z|00992|connmgr|INFO|br-int<->unix#515: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:44.821Z|00993|connmgr|INFO|br-int<->unix#518: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:45.299Z|00994|connmgr|INFO|br-int<->unix#521: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:45.809Z|00995|connmgr|INFO|br-int<->unix#524: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:46.310Z|00996|connmgr|INFO|br-int<->unix#527: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:46.805Z|00997|connmgr|INFO|br-int<->unix#530: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.185Z|00998|connmgr|INFO|br-int<->unix#533: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.567Z|00999|connmgr|INFO|br-int<->unix#536: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.987Z|01000|connmgr|INFO|br-int<->unix#539: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:48.417Z|01001|connmgr|INFO|br-int<->unix#542: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:48.865Z|01002|connmgr|INFO|br-int<->unix#545: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:49.408Z|01003|connmgr|INFO|br-int<->unix#548: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:49.918Z|01004|connmgr|INFO|br-int<->unix#551: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:50.399Z|01005|connmgr|INFO|br-int<->unix#554: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:50.871Z|01006|connmgr|INFO|br-int<->unix#557: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:51.337Z|01007|connmgr|INFO|br-int<->unix#560: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:51.627Z|01008|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 16 flow_mods in the 7 s starting 27 s ago (12 adds, 4 deletes) 2020-12-02T08:01:51.771Z|01009|connmgr|INFO|br-int<->unix#563: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:52.285Z|01010|connmgr|INFO|br-int<->unix#566: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:52.695Z|01011|connmgr|INFO|br-int<->unix#569: 96 flow_mods in the last 0 s (96 adds) 2. the AGENT_RES_PROCESSING_STEP is 1000 (changed) 12222 OpenFlow rules are installed in 6s. 2020-12-02 16:13:41.855 3090 INFO neutron.agent.common.ovs_lib [None req-0eef620d-613e-4485-ad2f-b9380c3fc381 - - - - -] Processing 12222 OpenFlow rules. 2020-12-02 16:13:47.846 3090 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-0eef620d-613e-4485-ad2f-b9380c3fc381 - - - - -] process_network_ports - iteration:0 - agent port security group processed in 202.523 2020-12-02T08:13:42.299Z|01310|connmgr|INFO|br-int<->unix#1298: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:42.718Z|01311|connmgr|INFO|br-int<->unix#1301: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:43.174Z|01312|connmgr|INFO|br-int<->unix#1304: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:43.698Z|01313|connmgr|INFO|br-int<->unix#1307: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:44.087Z|01314|connmgr|INFO|br-int<->unix#1310: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:44.554Z|01315|connmgr|INFO|br-int<->unix#1313: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.050Z|01316|connmgr|INFO|br-int<->unix#1316: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.534Z|01317|connmgr|INFO|br-int<->unix#1319: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.939Z|01318|connmgr|INFO|br-int<->unix#1322: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:46.388Z|01319|connmgr|INFO|br-int<->unix#1325: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:46.869Z|01320|connmgr|INFO|br-int<->unix#1328: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:47.323Z|01321|connmgr|INFO|br-int<->unix#1331: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:47.834Z|01322|connmgr|INFO|br-int<->unix#1334: 222 flow_mods in the last 0 s (222 adds) So we can say maybe we can increase the trunk (step) size for the ovs-ofctl bundle installaion. We do not want to add a fixed value because the vswithd may be in high pressure according to your deployment, the flow install performace may not be as good as your excepted. So we can add a config option for users, they can tune the size for their cloud based on the load of the agent, vswitchd and cpu usage and so on. We recently tested the openvswitch-agent flow installation performance, we have following two results: 1. the AGENT_RES_PROCESSING_STEP is 100 (default) We can see the 4296 OpenFlow rules are installed in 20s. 2020-12-02 16:01:32.843 792 INFO neutron.agent.common.ovs_lib [None req-106cf38e-e20a-443e-b1ad-24ab1b7dc7cb - - - - -] Processing 4296 OpenFlow rules. 2020-12-02 16:01:52.698 792 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-106cf38e-e20a-443e-b1ad-24ab1b7dc7cb - - - - -] process_network_ports - iteration:266 - agent port security group processed in 31.044 2020-12-02T08:01:32.303Z|00967|connmgr|INFO|br-int<->unix#440: 1 flow_mods in the last 0 s (1 adds) 2020-12-02T08:01:33.353Z|00968|connmgr|INFO|br-int<->unix#443: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:33.864Z|00969|connmgr|INFO|br-int<->unix#446: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:34.435Z|00970|connmgr|INFO|br-int<->unix#449: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:34.966Z|00971|connmgr|INFO|br-int<->unix#452: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:35.399Z|00972|connmgr|INFO|br-int<->unix#455: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:35.904Z|00973|connmgr|INFO|br-int<->unix#458: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:36.335Z|00974|connmgr|INFO|br-int<->unix#461: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:36.820Z|00975|connmgr|INFO|br-int<->unix#464: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:37.288Z|00976|connmgr|INFO|br-int<->unix#467: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:37.805Z|00977|connmgr|INFO|br-int<->unix#470: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:38.329Z|00978|connmgr|INFO|br-int<->unix#473: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:38.792Z|00979|connmgr|INFO|br-int<->unix#476: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:39.248Z|00980|connmgr|INFO|br-int<->unix#479: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:39.687Z|00981|connmgr|INFO|br-int<->unix#482: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.164Z|00982|connmgr|INFO|br-int<->unix#485: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.568Z|00983|connmgr|INFO|br-int<->unix#488: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:40.978Z|00984|connmgr|INFO|br-int<->unix#491: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:41.453Z|00985|connmgr|INFO|br-int<->unix#494: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:41.892Z|00986|connmgr|INFO|br-int<->unix#497: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:42.264Z|00987|connmgr|INFO|br-int<->unix#500: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:42.639Z|00988|connmgr|INFO|br-int<->unix#503: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.077Z|00989|connmgr|INFO|br-int<->unix#506: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.452Z|00990|connmgr|INFO|br-int<->unix#509: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:43.879Z|00991|connmgr|INFO|br-int<->unix#512: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:44.385Z|00992|connmgr|INFO|br-int<->unix#515: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:44.821Z|00993|connmgr|INFO|br-int<->unix#518: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:45.299Z|00994|connmgr|INFO|br-int<->unix#521: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:45.809Z|00995|connmgr|INFO|br-int<->unix#524: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:46.310Z|00996|connmgr|INFO|br-int<->unix#527: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:46.805Z|00997|connmgr|INFO|br-int<->unix#530: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.185Z|00998|connmgr|INFO|br-int<->unix#533: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.567Z|00999|connmgr|INFO|br-int<->unix#536: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:47.987Z|01000|connmgr|INFO|br-int<->unix#539: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:48.417Z|01001|connmgr|INFO|br-int<->unix#542: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:48.865Z|01002|connmgr|INFO|br-int<->unix#545: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:49.408Z|01003|connmgr|INFO|br-int<->unix#548: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:49.918Z|01004|connmgr|INFO|br-int<->unix#551: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:50.399Z|01005|connmgr|INFO|br-int<->unix#554: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:50.871Z|01006|connmgr|INFO|br-int<->unix#557: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:51.337Z|01007|connmgr|INFO|br-int<->unix#560: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:51.627Z|01008|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 16 flow_mods in the 7 s starting 27 s ago (12 adds, 4 deletes) 2020-12-02T08:01:51.771Z|01009|connmgr|INFO|br-int<->unix#563: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:52.285Z|01010|connmgr|INFO|br-int<->unix#566: 100 flow_mods in the last 0 s (100 adds) 2020-12-02T08:01:52.695Z|01011|connmgr|INFO|br-int<->unix#569: 96 flow_mods in the last 0 s (96 adds) 2. the AGENT_RES_PROCESSING_STEP is 1000 (changed) 12222 OpenFlow rules are installed in 6s. 2020-12-02 16:13:41.855 3090 INFO neutron.agent.common.ovs_lib [None req-0eef620d-613e-4485-ad2f-b9380c3fc381 - - - - -] Processing 12222 OpenFlow rules. 2020-12-02 16:13:47.846 3090 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-0eef620d-613e-4485-ad2f-b9380c3fc381 - - - - -] process_network_ports - iteration:0 - agent port security group processed in 202.523 2020-12-02T08:13:42.299Z|01310|connmgr|INFO|br-int<->unix#1298: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:42.718Z|01311|connmgr|INFO|br-int<->unix#1301: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:43.174Z|01312|connmgr|INFO|br-int<->unix#1304: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:43.698Z|01313|connmgr|INFO|br-int<->unix#1307: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:44.087Z|01314|connmgr|INFO|br-int<->unix#1310: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:44.554Z|01315|connmgr|INFO|br-int<->unix#1313: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.050Z|01316|connmgr|INFO|br-int<->unix#1316: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.534Z|01317|connmgr|INFO|br-int<->unix#1319: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:45.939Z|01318|connmgr|INFO|br-int<->unix#1322: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:46.388Z|01319|connmgr|INFO|br-int<->unix#1325: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:46.869Z|01320|connmgr|INFO|br-int<->unix#1328: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:47.323Z|01321|connmgr|INFO|br-int<->unix#1331: 1000 flow_mods in the last 0 s (1000 adds) 2020-12-02T08:13:47.834Z|01322|connmgr|INFO|br-int<->unix#1334: 222 flow_mods in the last 0 s (222 adds) So we can say maybe we should increase the trunk (step) size for the ovs-ofctl bundle installaion. We do not want to add a fixed value because the vswitchd may be in high pressure according to your deployment, the flow install performace may not be as good as you expected. So we could add a config option for users, they can tune the size for their cloud based on the load of the agent, vswitchd and cpu usage and so on.
2020-12-02 09:36:19 LIU Yulong neutron: status New In Progress
2020-12-03 08:54:38 Bernard Cafarelli neutron: importance Undecided Medium
2021-01-04 12:07:22 Slawek Kaplonski tags loadimpact ovs