I have tested with the suggested upstream v4.6 kernel, running up to 12000 tcp conn/s without hitting the problem.
# uname -a Linux ovs1604 4.6.0-040600rc7-generic #201605081830 SMP Sun May 8 22:32:57 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
As a side note I was also able to reproduce the problem with the 4.4.9-040409 kernel.
I have tested with the suggested upstream v4.6 kernel, running up to 12000 tcp conn/s without hitting the problem.
# uname -a -generic #201605081830 SMP Sun May 8 22:32:57 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Linux ovs1604 4.6.0-040600rc7
As a side note I was also able to reproduce the problem with the 4.4.9-040409 kernel.