Send all flow info in all exports of the flow

[Ashok Singh]

Currently vrouter-agent sends the following fields only once

setup-time
vrouter-ip
peer-vrouter-ip
underlay-port
tunnel-type

With flow sampling in place, if flow sample with above information is dropped, the rest of the samples for this flow, which make it to collector will never have this info. To overcome this, it was agreed to send all the flow fields always regardless of whether any field was previously sent or not.

Also it was agreed to remove teardown-time since it is sent only once for delete and there are chances of this sample getting dropped. Since we cannot guarantee teardown-time for a flow to be always sent, it was agreed to be removed.

Also remove unused flow fields from flow.sandesh.

Email discussion on this topic :

From: Raj Reddy <email address hidden>
Date: Friday, January 15, 2016 at 3:18 AM
To: Ashok Singh R <email address hidden>
Cc: Sundaresan Rajangam <email address hidden>, Anish Mehta <email address hidden>, Hari Prasad Killi <email address hidden>
Subject: Re: Flow fields not being populated by agent

Hi Ashok,

You can remove the below and teardown_time, let’s keep agg packets/bytes..
You can go ahead and commit changes for sending all fields in every sample..

There’s no dependency on the analytics changes, there’s some clean up required, and I will do it after your commit.

thanks,
-Raj

On Jan 12, 2016, at 11:23 PM, Ashok Singh R <email address hidden> wrote:

Hi Raj,

I pasted diff of flow.sandesh containing fields which are not being used by agent.

ashoksr@nodeb6:~/mainline/controller$ git diff
diff --git a/src/sandesh/common/flow.sandesh b/src/sandesh/common/flow.sandesh
index 6b38e55..0c11e10 100644
--- a/src/sandesh/common/flow.sandesh
+++ b/src/sandesh/common/flow.sandesh
@@ -18,24 +18,14 @@ struct FlowDataIpv4 {
     7: optional byte protocol;
     8: optional i16 sport;
     9: optional i16 dport;
- 10: optional byte tos;
     11: optional u16 tcp_flags;
     12: optional string vm;
- 13: optional string input_interface;
- 14: optional string output_interface;
- 15: optional i32 mpls_label;
     16: optional string reverse_uuid;
     17: optional i64 setup_time;
     18: optional i64 teardown_time;

- 19: optional i32 min_interarrival;
- 20: optional i32 max_interarrival;
- 21: optional i32 mean_interarrival;
- 22: optional i32 stddev_interarrival;
-
     23: optional i64 bytes;
     24: optional i64 packets;
- 25: optional binary data_sample;
     26: optional i64 diff_bytes;
     27: optional i64 diff_packets;
     28: optional string action;

On top of these the following are the fields which are being used by agent, but considered for removal

teardown_time
Bytes (aggregate bytes)
Packets (aggregate packets)
Please summarize all the agent changes required. I will make all of them in 1 commit. As of now only thing agreed on last night’s call is to send all the flow fields of agent always

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16361
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16361
Committed: http://github.org/Juniper/contrail-controller/commit/958d5bd4eeac276d16d5b188009c032b1edcdc94
Submitter: Zuul
Branch: master

commit 958d5bd4eeac276d16d5b188009c032b1edcdc94
Author: Ashok Singh <email address hidden>
Date: Wed Jan 20 20:28:02 2016 -0800

Send all fields of flow always in FlowLog messages

Currently vrouter-agent sends the following fields only once

setup-time
vrouter-ip
peer-vrouter-ip
underlay-port
tunnel-type

With flow sampling in place, if flow sample with above information is dropped,
the rest of the samples for this flow, which make it to collector will never
have this info. To overcome this we send all the fields always.

Also remove teardown-time since it is sent only once for delete and there are
chances of this sample getting dropped

Also remove unused flow fields from flow.sandesh.

Change-Id: Ib17fad30bc9d859375f82de363e57f97251686db
Closes-Bug: #1536118