flow export rate much higher than what is configured

[Vedamurthy Joshi]

R3.0.2.0 build 41

In this setup, there are 13 vrouters

flow_export_rate was set to 10 flows/s

It was seen that collector was seeing an average of instead of 327K flows/min 7.8K flows/min(13 vrouter *10 flows/s *60 sec)

i.e. about 400 flows/s from each vrouter on an average

contrail-stats --table SandeshMessageStat.msg_info --select "SUM(msg_info.messages)" msg_info.type --last 1m | grep Flow

{u'msg_info.type': u'FlowLogDataObject', u'SUM(msg_info.messages)': 327252}

[Ashok Singh (ashoksr)]

When continuous traffic is flowing, the flow-export-rate computed every 2 seconds along with their threshold is mentioned below. The configured flow-export-rate in the system is 100.

2017-05-31 Tue 14:08:59:368.777 IST : iter 21 exp-rate 3647 old_t 500 new_t 2000
2016-05-31 Tue 14:09:01:370.429 IST : iter 22 exp-rate 454 old_t 2000 new_t 8000
2016-05-31 Tue 14:09:03:370.730 IST : iter 23 exp-rate 246 old_t 8000 new_t 24000
2016-05-31 Tue 14:09:05:415.050 IST : iter 24 exp-rate 125 old_t 24000 new_t 24000
2016-05-31 Tue 14:09:07:435.309 IST : iter 25 exp-rate 184 old_t 24000 new_t 48000
2016-05-31 Tue 14:09:09:452.037 IST : iter 26 exp-rate 133 old_t 48000 new_t 96000
2016-05-31 Tue 14:09:11:468.498 IST : iter 27 exp-rate 78 old_t 96000 new_t 500
2016-05-31 Tue 14:09:13:489.563 IST : iter 28 exp-rate 6781 old_t 500 new_t 2000
2016-05-31 Tue 14:09:15:540.984 IST : iter 29 exp-rate 2019 old_t 2000 new_t 8000
2016-05-31 Tue 14:09:17:551.558 IST : iter 30 exp-rate 573 old_t 8000 new_t 32000
2016-05-31 Tue 14:09:19:579.443 IST : iter 31 exp-rate 188 old_t 32000 new_t 64000
2016-05-31 Tue 14:09:21:583.768 IST : iter 32 exp-rate 164 old_t 64000 new_t 128000
2016-05-31 Tue 14:09:23:611.633 IST : iter 33 exp-rate 136 old_t 128000 new_t 256000
2016-05-31 Tue 14:09:25:612.670 IST : iter 34 exp-rate 95 old_t 256000 new_t 256000
2016-05-31 Tue 14:09:27:613.367 IST : iter 35 exp-rate 106 old_t 256000 new_t 256000
2016-05-31 Tue 14:09:29:613.523 IST : iter 36 exp-rate 104 old_t 256000 new_t 256000
2016-05-31 Tue 14:09:31:619.266 IST : iter 37 exp-rate 110 old_t 256000 new_t 256000
2016-05-31 Tue 14:09:33:619.403 IST : iter 38 exp-rate 140 old_t 256000 new_t 512000
2016-05-31 Tue 14:09:35:622.764 IST : iter 39 exp-rate 79 old_t 512000 new_t 500
2016-05-31 Tue 14:09:37:635.885 IST : iter 40 exp-rate 7323 old_t 500 new_t 2000
2016-05-31 Tue 14:09:39:641.540 IST : iter 41 exp-rate 6635 old_t 2000 new_t 8000
2016-05-31 Tue 14:09:41:649.478 IST : iter 42 exp-rate 2329 old_t 8000 new_t 32000
2016-05-31 Tue 14:09:43:656.681 IST : iter 43 exp-rate 394 old_t 32000 new_t 128000
2016-05-31 Tue 14:09:45:657.609 IST : iter 44 exp-rate 103 old_t 128000 new_t 128000
2016-05-31 Tue 14:09:47:672.258 IST : iter 45 exp-rate 137 old_t 128000 new_t 256000
2016-05-31 Tue 14:09:49:698.808 IST : iter 46 exp-rate 92 old_t 256000 new_t 256000
2016-05-31 Tue 14:09:51:729.935 IST : iter 47 exp-rate 62 old_t 256000 new_t 500
2016-05-31 Tue 14:09:53:736.742 IST : iter 48 exp-rate 6789 old_t 500 new_t 2000
2016-05-31 Tue 14:09:55:747.590 IST : iter 49 exp-rate 4243 old_t 2000 new_t 8000
2016-05-31 Tue 14:09:57:794.403 IST : iter 50 exp-rate 2565 old_t 8000 new_t 32000
2016-05-31 Tue 14:09:59:800.040 IST : iter 51 exp-rate 477 old_t 32000 new_t 128000
2016-05-31 Tue 14:10:01:800.463 IST : iter 52 exp-rate 194 old_t 128000 new_t 256000
2016-05-31 Tue 14:10:03:802.773 IST : iter 53 exp-rate 84 old_t 256000 new_t 256000
2016-05-31 Tue 14:10:05:802.938 IST : iter 54 exp-rate 108 old_t 256000 new_t 256000
2016-05-31 Tue 14:10:07:810.788 IST : iter 55 exp-rate 122 old_t 256000 new_t 256000
2016-05-31 Tue 14:10:09:810.944 IST : iter 56 exp-rate 111 old_t 25600...

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23211
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23210
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23210
Committed: http://github.org/Juniper/contrail-controller/commit/fb9fa3f0579792518d76dbf24dd7aeb9bdfdc208
Submitter: Zuul
Branch: master

commit fb9fa3f0579792518d76dbf24dd7aeb9bdfdc208
Author: Ashok Singh <email address hidden>
Date: Thu Aug 11 04:58:38 2016 -0700

Flow Sampling changes

1.Do gradual reduction of flow-threshold when flow-export-rate has come
down after reaching a high value (instead of resetting it to default threshold)
2.Do not disable sampling of flows when flow-export-rate drops to below 80% of
configured rate after reaching high values
3.Define a minimum threshold value 20
4.Define a tracebuffer to log flow-export-rates and thresholds

Also removed unused variable in FlowStatsCollector object.

Change-Id: I74e7baa84f2792bc451804fda350a75f817d1f6e
Partial-Bug: #1586246

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23211
Committed: http://github.org/Juniper/contrail-controller/commit/0271ce169af1db2d626c5615b5532cd0bb50ed9b
Submitter: Zuul
Branch: R3.1

commit 0271ce169af1db2d626c5615b5532cd0bb50ed9b
Author: Ashok Singh <email address hidden>
Date: Thu Aug 11 04:58:38 2016 -0700

Flow Sampling changes

1.Do gradual reduction of flow-threshold when flow-export-rate has come
down after reaching a high value (instead of resetting it to default threshold)
2.Do not disable sampling of flows when flow-export-rate drops to below 80% of
configured rate after reaching high values
3.Define a minimum threshold value 20
4.Define a tracebuffer to log flow-export-rates and thresholds

Also removed unused variable in FlowStatsCollector object.

Partial-Bug: #1586246
(cherry picked from commit b541d966fe0f15c1ded0016ab19aa8077cb41159)

Change-Id: I527f59b5b7b707252ec412cceded94094ee8bd54

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23353
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23353
Committed: http://github.org/Juniper/contrail-controller/commit/60256bfabe1ace9d9294138a93c6d20b0c60c9b7
Submitter: Zuul
Branch: R3.0

commit 60256bfabe1ace9d9294138a93c6d20b0c60c9b7
Author: Ashok Singh <email address hidden>
Date: Thu Aug 11 04:58:38 2016 -0700

Flow Sampling changes

1.Do gradual reduction of flow-threshold when flow-export-rate has come
down after reaching a high value (instead of resetting it to default threshold)
2.Do not disable sampling of flows when flow-export-rate drops to below 80% of
configured rate after reaching high values
3.Define a minimum threshold value 20
4.Define a tracebuffer to log flow-export-rates and thresholds

Also removed unused variable in FlowStatsCollector object.

Partial-Bug: #1586246
(cherry picked from commit fb9fa3f0579792518d76dbf24dd7aeb9bdfdc208)

Change-Id: Ifac0ca2a4b6daa1de80682fa9e6030bbb8216c7c