Memcache clients not handling memcache restarts correctly

Bug #724558 reported by David Goetz on 2011-02-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Undecided
David Goetz

Bug Description

When memcache was restarted the ratelimiting allowed puts at double the listed rate. It seems that the proxies were talking to two different memcache servers.

Related branches

Changed in swift:
assignee: nobody → David Goetz (david-goetz)
David Goetz (david-goetz) wrote :

On restarts, a memcached server will not respond for a short period of time. During that time, proxy processes talking to that server will error limit it and start using the next memcached server in the ring. What happens is that during the short period of time that the memcached is restarted not all of the proxy processes will attempt to make a connection enough times to error limit the memcached server. The result of this is that different proxy processes will be using different memcached servers and services that rely on them all talking to the same memcached server will not work correctly. In the case of ratelimiting, the rate allowed can double. The fix for this for now is just to decrease the amount of time the memcached server is error limited from 5 minutes to 1 minute.

Changed in swift:
status: New → Fix Committed
Thierry Carrez (ttx) on 2011-04-15
Changed in swift:
milestone: none → 1.3.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers