init.d restart operation can fail without proper notice if the memcached process takes a long time (> 1s) to shut down

Bug #795673 reported by trent strong on 2011-06-10
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
memcached (Debian)
Fix Released
Unknown
memcached (Ubuntu)
Medium
Clint Byrum
Lucid
Medium
Unassigned

Bug Description

TEST CASE:

1. install memcached and python-memcache
2. edit /etc/memcached.conf and change -m 64 to -m 640
3. sudo service memcached restart
4. Run the following test script with '300000' as an argument

#!/usr/bin/python

import memcache
import sys

mc = memcache.Client(['127.0.0.1:11211'], debug=1)

i=0
while i < int(sys.argv[1]):
    mc.set("key_%09d" % i, "A" * 1024)
    i += 1

5. Run the script again, with 1000000 with & to background it
6. sudo service memcached restart
7. verify that memcached is running (pidof memcached). If it is not, then this bug exists.

As this is a race condition, you may have to repeat steps 4 through 7 a few times before you reproduce the issue. On precise, it took about 15 tries. If you have the older version of /etc/init.d/memcached which does 'sleep 1' in it, then you can make it more likely by removing the sleep 1.

----

Binary package hint: memcached

I have noticed that sometimes on a production memcached service the restart command is not always reliable. If the memcached process has allocated a lot of memory, it sometimes take a little while to shut down.

The init.d script seems to approach this by inserting a "sleep 1" between the stop and start operations, but that's not always enough.
When the memcached service is started again it will fail to start with the error "failed to listen on TCP port XXXXX: Address already in use" since the dying process still owns the port.

Release: Ubuntu 10.10

memcached:
  Installed: 1.4.5-1ubuntu1
  Candidate: 1.4.5-1ubuntu1
  Version table:
 *** 1.4.5-1ubuntu1 0
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ maverick/main amd64 Packages
        100 /var/lib/dpkg/status

Related branches

Clint Byrum (clint-fewbar) wrote :

I've seen this as well trent and never gotten around to reporting it. Marking Confirmed, setting importance to Medium.

Changed in memcached (Ubuntu):
status: New → Confirmed
importance: Undecided → Medium
Changed in memcached (Ubuntu):
status: Confirmed → In Progress
assignee: nobody → Clint Byrum (clint-fewbar)
description: updated
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package memcached - 1.4.13-0ubuntu1

---------------
memcached (1.4.13-0ubuntu1) precise; urgency=low

  * New upstream release.
  * Refreshed patches
  * d/p/50_add_init_retry.patch: use --retry to wait up to 5 seconds
    for memcached to die. (LP: #795673)
  * d/rules: run test suite on build (LP: #586632)
 -- Clint Byrum <email address hidden> Thu, 09 Feb 2012 15:35:15 -0800

Changed in memcached (Ubuntu):
status: In Progress → Fix Released
Changed in memcached (Ubuntu Lucid):
status: New → Triaged
importance: Undecided → Medium
Changed in memcached (Debian):
status: Unknown → New
Changed in memcached (Debian):
status: New → Fix Committed
Changed in memcached (Debian):
status: Fix Committed → Fix Released
Rolf Leggewie (r0lf) wrote :

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".

Changed in memcached (Ubuntu Lucid):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.