3.1.1-45:flow -s reports inconsistencies

Bug #1654941 reported by Sandeep Sridhar
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
Medium
Kumar Harsh
R3.1
Fix Committed
Medium
Kumar Harsh
R3.2
Fix Committed
Medium
Kumar Harsh
Trunk
Fix Committed
Medium
Kumar Harsh

Bug Description

Contrail Version: 3.1.1-Build 45.

flow -s reports inconsistencies when there are high number of flows on the vRouter (simulated in the lab through hping3)

Average Setup/Teardown rate in flow shows a very high value at times.

Please see below:
2017-01-02 15:04:29 +0900
Flow Statistics
---------------
Total Entries --- Total = 986581, new = 11032
Active Entries --- Total = 979447, new = 10780
Hold Entries --- Total = 7134, new = 252
Fwd flow Entries - Total = 979447
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 19779
Avg setup rate = 18565
Avg teardown rate = 20397
Rate of change of Flow Entries
------------------------------
current rate = 20242
2017-01-02 15:04:30 +0900
Flow Statistics
---------------
Total Entries --- Total = 998368, new = 11787
Active Entries --- Total = 992736, new = 13289
Hold Entries --- Total = 5632, new = -1502
Fwd flow Entries - Total = 992736
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 24473
Avg setup rate = 2092458556 <<<<<
Avg teardown rate = 20397
Rate of change of Flow Entries
------------------------------
current rate = 21707
(snip)
2017-01-02 15:10:40 +0900
Flow Statistics
---------------
Total Entries --- Total = 0, new = 0
Active Entries --- Total = 0, new = 0
Hold Entries --- Total = 0, new = 0
Fwd flow Entries - Total = 0
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 0
Avg setup rate = 760246994 <<<<<<
Avg teardown rate = -836050317 <<<<<<
Rate of change of Flow Entries
------------------------------
current rate = 0

Also, the logic to calculate "current rate" and "avg setup rate" from the incremental difference of Active entries. I guess, the code should skip the calculation of current rate for the first time since previous value makes sense only from the second time.

============================================================================
2017-01-02 15:13:23 +0900
Flow Statistics
---------------
Total Entries --- Total = 1169992, new = 0
Active Entries --- Total = 1169992, new = 0
Hold Entries --- Total = 0, new = 0
Fwd flow Entries - Total = 1169992
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 0
Avg setup rate = 17154
Avg teardown rate = 0
Rate of change of Flow Entries
------------------------------
current rate = 0
^C
root@sv-9:~# flow -s
2017-01-02 15:13:26 +0900
Flow Statistics
---------------
Total Entries --- Total = 1169992, new = 1169992
Active Entries --- Total = 1169992, new = 1169992
Hold Entries --- Total = 0, new = 0
Fwd flow Entries - Total = 1169992
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 1914880 <<<<<
Avg setup rate = 1914880 <<<<<
Avg teardown rate = 0
Rate of change of Flow Entries
------------------------------
current rate = 1914880
2017-01-02 15:13:26 +0900
Flow Statistics
---------------
Total Entries --- Total = 1169992, new = 0
Active Entries --- Total = 1169992, new = 0
Hold Entries --- Total = 0, new = 0
Fwd flow Entries - Total = 1169992
drop flow Entries - Total = 0
NAT flow Entries - Total = 0

Rate of change of Active Entries
--------------------------------
current rate = 0
Avg setup rate = 1914880
Avg teardown rate = 0
Rate of change of Flow Entries
------------------------------
current rate = 0

As these values are picked from the kernel mode (vRouter), please fix this issue in the next builds.

Greetings,
Sandeep.

information type: Proprietary → Public
tags: added: vrouter
Changed in juniperopenstack:
assignee: nobody → Kumar Harsh (hkumar)
importance: Undecided → Medium
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/27925
Submitter: Kumar Harsh (<email address hidden>)

tags: added: vrouter3.1-45
removed: vrouter
tags: added: vrouter
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/27996
Submitter: Kumar Harsh (<email address hidden>)

Revision history for this message
Kumar Harsh (hkumar) wrote :

Undefined behaviour in flow -s is seen due to overflow of int32 variable.

From now flow -s will show flow setup calculations :
current rata =0
Avg setup rate =0
Avg teardown rate=0

for first time even if flows exist

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/27996
Submitter: Kumar Harsh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/28000
Submitter: Kumar Harsh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/28001
Submitter: Kumar Harsh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/28002
Submitter: Kumar Harsh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/27996
Committed: http://github.org/Juniper/contrail-vrouter/commit/8cc1dc61b602ff1ff891d32eb0c03de6f5007137
Submitter: Zuul (<email address hidden>)
Branch: master

commit 8cc1dc61b602ff1ff891d32eb0c03de6f5007137
Author: Kumar Harsh <email address hidden>
Date: Thu Jan 19 14:16:56 2017 +0530

Make average setup rate as int64 to prevent overflow in calculation
with high number of flows.
Skip flow setup rate calculations for the first time.

Change-Id: Ib17bda69b7e1994a460512035be09888ad78500c
Closes-Bug:#1654941

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/28000
Committed: http://github.org/Juniper/contrail-vrouter/commit/e922e19055c5221fa791cd2f65974a3592cfbd7f
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit e922e19055c5221fa791cd2f65974a3592cfbd7f
Author: Kumar Harsh <email address hidden>
Date: Thu Jan 19 15:50:28 2017 +0530

Make average setup rate as int64 to prevent overflow in calculation
with high number of flows.
Skip flow setup rate calculations for the first time.

Change-Id: I45297db8482685a52c69b6253a06c8c0ea5bb24b
Closes-Bug:#1654941

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/28002
Committed: http://github.org/Juniper/contrail-vrouter/commit/3cd15860c7f9ab54b16cec9f497a088609d0512a
Submitter: Zuul (<email address hidden>)
Branch: R3.0

commit 3cd15860c7f9ab54b16cec9f497a088609d0512a
Author: Kumar Harsh <email address hidden>
Date: Thu Jan 19 14:16:56 2017 +0530

Make average setup rate as uint64 to prevent overflow in calculation
with high number of flows.
Skip flow setup rate calculations for the first time.

Change-Id: Ib17bda69b7e1994a460512035be09888ad78500c
Closes-Bug:#1654941

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/28001
Committed: http://github.org/Juniper/contrail-vrouter/commit/465d5dd7a435576eed285c597cfcaed0fd91ef41
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 465d5dd7a435576eed285c597cfcaed0fd91ef41
Author: Kumar Harsh <email address hidden>
Date: Thu Jan 19 14:16:56 2017 +0530

Make average setup rate as uint64 to prevent overflow in calculation
with high number of flows.
Skip flow setup rate calculations for the first time.

Change-Id: Ib17bda69b7e1994a460512035be09888ad78500c
Closes-Bug:#1654941

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.