flow export rate much less than expected

[Vedamurthy Joshi]

R3.2.2.0 Build 31

Set flow export rate to 10K
From 10.1.1.3(VM) run hping to 10.1.1.4(VM) so that 100K tcp streams are created and evicted at 1Kpps for 100 secs

hping3 --count 100000 --baseport 1000 --interval u1000 --destport 22 10.1.1.4

It is seen that only <20K flows are seen in collector as exported flows

root@nodei31:/var/log/contrail# date; contrail-stats --table SandeshMessageStat.msg_info --select "SUM(msg_info.messages)" --last 10m --where 'name=nodei31:Compute:contrail-vrouter-agent:0' 'msg_info.type=FlowLogDataObject'
Thu Mar 23 14:26:19 IST 2017
{"start_time": "now-10m", "sort_fields": [], "end_time": "now", "select_fields": ["SUM(msg_info.messages)"], "table": "StatTable.SandeshMessageStat.msg_info", "where": [[{"suffix": null, "value2": null, "name": "name", "value": "nodei31:Compute:contrail-vrouter-agent:0", "op": 1}, {"suffix": null, "value2": null, "name": "msg_info.type", "value": "FlowLogDataObject", "op": 1}]]}
{u'SUM(msg_info.messages)': 22233}

[Raj Reddy (rajreddy)]

Ashok, can you take a look from agent side. may be it's just that number of active flows itself is low..

[Vedamurthy Joshi (vedujoshi)]

The right query to be done is
res = vns.post_query(
            'FlowSeriesTable',
            start_time=str(generator_obj.flow_start_time),
            end_time=str(generator_obj.flow_end_time),
            select_fields=['sum(bytes)', 'sum(packets)', 'flow_count'],
            where_clause='vrouter=%s'% vrouter)

But with that too, for 10K tcp syn packets(20K flows) at 1K pps, analytics showed only about 13-15K flows
If i send 1K tcp syn packets(2K flows) at 100 pps, all 2K flows are seen in analytics
With Ashok's binary, http://nodei31:8085/Snh_AgentStatsReq? showed that all flows are exported,
but there some drops seen in sandesh client stats http://nodei31:8085/Snh_SandeshUVECacheReq?x=ModuleClientState

[Anish Mehta (amehta00)]

Please retest after flow sampling is committed as part of Session Infra changes of R4.1