Flows not created intermittently

[Praveen]

1. It was observed that many compute nodes have flows in HOLD state. Ideally, HOLD state should only be a temporary state. When Vrouter kernel module identifies a new flow, it allocates an entry from flow-table puts it in HOLD state and sends message to Agent for flow setup. Once agent programs the flow the action moves from HOLD to FORWARD/DENY/etc… as appropriate.
2. It was observed that randomly new flow sessions do not get setup.
     1. Vrouter kernel module is receiving packets
     2. No flows are being created by Vrouter module
     3. No packets trapped on pkt0 for flow-setup
     4. No logs in agent trace-buffer for flows
     5. Vrouter dropstats show that “Flow Unusable” keeps incrementing
3. If flows in HOLD state deleted, the session gets setup and works fine. Note, the flows in HOLD state are not related to the new flow being setup.

The “Flow Unsable” counter is incremented when number of flows in HOLD state exceed a limit in Vrouter.

Based on the observations, the current suspect is accounting of flows in HOLD state has gone bad. As a result new flows are not being setup. Once flows in HOLD state are manually deleted the flows resume.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/8881
Submitter: Anand H. Krishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/8881
Committed: http://github.org/Juniper/contrail-vrouter/commit/3dbe5b528411add85b8ebcca356d998e7e854fde
Submitter: Zuul
Branch: R1.10

commit 3dbe5b528411add85b8ebcca356d998e7e854fde
Author: Anand H. Krishnan <email address hidden>
Date: Fri Apr 3 16:16:03 2015 +0530

Return error to agent if an entry already existed in the flow table

It is logically possible that agent and datapath are trying to create
same flow simultaneously. If it so happens that agent gets the entry
that datapath created and tries to update that entry assuming that the
entry was created by it, then the hold count will never be compensated
by a corresponding acted count, and hence vrouter's perception of the
number of active hold entries can go wrong. To fix this, return error
to agent if the flow it tried to create already existed.

Other fixes:

. If agent is changing the flow state to 'hold' from any other state,
update the hold count entry.

. Export the hold count statistics to 'flow' utility

Change-Id: I24087baa5bf853b863f34e1b55882927d9114349
Partial-BUG: #1439069

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/8946
Submitter: Anand H. Krishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/8948
Submitter: Anand H. Krishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/8946
Committed: http://github.org/Juniper/contrail-vrouter/commit/a2fde20b47a73ec1cea6219f19f41c5894cb007b
Submitter: Zuul
Branch: R2.0

commit a2fde20b47a73ec1cea6219f19f41c5894cb007b
Author: Anand H. Krishnan <email address hidden>
Date: Fri Apr 3 16:16:03 2015 +0530

Return error to agent if an entry already existed in the flow table

It is logically possible that agent and datapath are trying to create
same flow simultaneously. If it so happens that agent gets the entry
that datapath created and tries to update that entry assuming that the
entry was created by it, then the hold count will never be compensated
by a corresponding acted count, and hence vrouter's perception of the
number of active hold entries can go wrong. To fix this, return error
to agent if the flow it tried to create already existed.

Other fixes:

. If agent is changing the flow state to 'hold' from any other state,
update the hold count entry.

. Export the hold count statistics to 'flow' utility

Change-Id: I24087baa5bf853b863f34e1b55882927d9114349
Partial-BUG: #1439069

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/8948
Committed: http://github.org/Juniper/contrail-vrouter/commit/f199a6b3e58789fe337134c7c32a27565dabb614
Submitter: Zuul
Branch: master

commit f199a6b3e58789fe337134c7c32a27565dabb614
Author: Anand H. Krishnan <email address hidden>
Date: Fri Apr 3 16:16:03 2015 +0530

Return error to agent if an entry already existed in the flow table

It is logically possible that agent and datapath are trying to create
same flow simultaneously. If it so happens that agent gets the entry
that datapath created and tries to update that entry assuming that the
entry was created by it, then the hold count will never be compensated
by a corresponding acted count, and hence vrouter's perception of the
number of active hold entries can go wrong. To fix this, return error
to agent if the flow it tried to create already existed.

Other fixes:

. If agent is changing the flow state to 'hold' from any other state,
update the hold count entry.

. Export the hold count statistics to 'flow' utility

Change-Id: I24087baa5bf853b863f34e1b55882927d9114349
Partial-BUG: #1439069

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/9146
Submitter: Anand H. Krishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/9146
Committed: http://github.org/Juniper/contrail-vrouter/commit/be6be5911040e338bfd9422be1b414f2c8e9b56b
Submitter: Zuul
Branch: R2.1

commit be6be5911040e338bfd9422be1b414f2c8e9b56b
Author: Anand H. Krishnan <email address hidden>
Date: Fri Apr 3 16:16:03 2015 +0530

Return error to agent if an entry already existed in the flow table

It is logically possible that agent and datapath are trying to create
same flow simultaneously. If it so happens that agent gets the entry
that datapath created and tries to update that entry assuming that the
entry was created by it, then the hold count will never be compensated
by a corresponding acted count, and hence vrouter's perception of the
number of active hold entries can go wrong. To fix this, return error
to agent if the flow it tried to create already existed.

Other fixes:

. If agent is changing the flow state to 'hold' from any other state,
update the hold count entry.

. Export the hold count statistics to 'flow' utility

Change-Id: I24087baa5bf853b863f34e1b55882927d9114349
Partial-BUG: #1439069