Debugged this further and found that the bug happens if the BGP VM goes away without actually FIN/RSTing the TCP flow. Now, if the VM tries to establish a new TCP connection to the same destination, the flow gets stuck in Hold state and packets are silently discarded.
The right way to handle this would be to forward the new SYN to the CN and updating the VM->vRouter side of the flow with the new source port. Once the SYN hits the CN, nne of the two scenarios would play out:
1. If syn.seq is in window, CN tcp stack will respond back with a RST. This RST should be forwarded to the VM.
or
2. if syn.seq is out of the window, the CN tcp stack will respond back with an ACK (in an attempt to resynchronize the client). The BGP VM on seeing this ACK will RST the connection (doesn't expect ACK with wrong seq in SYN-sent state), which should be forwarded back to the CN. The RST.seq will be the same as ACK.ack, and so the CN tcp state will be cleaned up.
Either way, the idea is to end up synchronizing the source and destination hosts on the state of the TCP flow; in this case, it should be to abort the old flow and start a new one. After scenario 1/2 plays out, the BGP VM will be able to successfully setup a new peering session with the CN on the next attempt.
Debugged this further and found that the bug happens if the BGP VM goes away without actually FIN/RSTing the TCP flow. Now, if the VM tries to establish a new TCP connection to the same destination, the flow gets stuck in Hold state and packets are silently discarded.
The right way to handle this would be to forward the new SYN to the CN and updating the VM->vRouter side of the flow with the new source port. Once the SYN hits the CN, nne of the two scenarios would play out:
1. If syn.seq is in window, CN tcp stack will respond back with a RST. This RST should be forwarded to the VM.
or
2. if syn.seq is out of the window, the CN tcp stack will respond back with an ACK (in an attempt to resynchronize the client). The BGP VM on seeing this ACK will RST the connection (doesn't expect ACK with wrong seq in SYN-sent state), which should be forwarded back to the CN. The RST.seq will be the same as ACK.ack, and so the CN tcp state will be cleaned up.
Either way, the idea is to end up synchronizing the source and destination hosts on the state of the TCP flow; in this case, it should be to abort the old flow and start a new one. After scenario 1/2 plays out, the BGP VM will be able to successfully setup a new peering session with the CN on the next attempt.