Never reconnect after connection reset (binary protocol)

[Martin C. Martin]

When I restart my memcached server, libmemcached never reconnects as long as I'm continuously trying to send data to it. If the client stops activity for a few seconds then starts sending again, the reconnection happens just fine.

It turns out to be a problem with how next_rety is set. Even when we don't try to connect because it's too soon, we still update next_retry.

That's because the socket is invalid, so the server state is MEMCACHED_SERVER_STATE_NEW, not MEMCACHED_SERVER_STATE_IN_TIMEOUT. memcached_mark_server_for_timeout() only skips updating next_retry in the latter state, not the former.

I'm not sure what the right solution is here. Maybe make memcached_quit_server() idempotent by checking for MEMCACHED_SERVER_STATE_NEW at the start and returing right away? I'm reluctant to do the same in memcached_mark_server_for_timeout(), in case that skips setting it when we really do need to set it.

[Martin C. Martin (martin-b69y0hv8h)]

Simply checking for MEMCACHED_SERVER_STATE_NEW in memcached_quit_server() won't work, because it's already in MEMCACHED_SERVER_STATE_IN_TIMEOUT.

Perhaps the problem is in memcached_send_binary. memcached_send_ascii has different logic here. When it writes the header, if we don't want a reply or flush we return right away. Otherwise, we only call memcached_io_reset if the return code is MEMCACHED_WRITE_FAILURE. Should the binary protocl use the same behavior? Perhaps there's a refactoring where we can write the logic once and use it for both the binary and ascii protocols, but that can be left to another time.

[Martin C. Martin (martin-b69y0hv8h)]

Follows the behavior of the ascii send.