init.d restart operation can fail without proper notice if the memcached process takes a long time (> 1s) to shut down

[trent strong]

TEST CASE:

1. install memcached and python-memcache
2. edit /etc/memcached.conf and change -m 64 to -m 640
3. sudo service memcached restart
4. Run the following test script with '300000' as an argument

#!/usr/bin/python

import memcache
import sys

mc = memcache.Client(['127.0.0.1:11211'], debug=1)

i=0
while i < int(sys.argv[1]):
    mc.set("key_%09d" % i, "A" * 1024)
    i += 1

5. Run the script again, with 1000000 with & to background it
6. sudo service memcached restart
7. verify that memcached is running (pidof memcached). If it is not, then this bug exists.

As this is a race condition, you may have to repeat steps 4 through 7 a few times before you reproduce the issue. On precise, it took about 15 tries. If you have the older version of /etc/init.d/memcached which does 'sleep 1' in it, then you can make it more likely by removing the sleep 1.

----

Binary package hint: memcached

I have noticed that sometimes on a production memcached service the restart command is not always reliable. If the memcached process has allocated a lot of memory, it sometimes take a little while to shut down.

The init.d script seems to approach this by inserting a "sleep 1" between the stop and start operations, but that's not always enough.
When the memcached service is started again it will fail to start with the error "failed to listen on TCP port XXXXX: Address already in use" since the dying process still owns the port.

Release: Ubuntu 10.10

memcached:
  Installed: 1.4.5-1ubuntu1
  Candidate: 1.4.5-1ubuntu1
  Version table:
 *** 1.4.5-1ubuntu1 0
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ maverick/main amd64 Packages
        100 /var/lib/dpkg/status

[Clint Byrum (clint-fewbar)]

I've seen this as well trent and never gotten around to reporting it. Marking Confirmed, setting importance to Medium.

[Launchpad Janitor (janitor)]

This bug was fixed in the package memcached - 1.4.13-0ubuntu1

---------------
memcached (1.4.13-0ubuntu1) precise; urgency=low

  * New upstream release.
  * Refreshed patches
  * d/p/50_add_init_retry.patch: use --retry to wait up to 5 seconds
    for memcached to die. (LP: #795673)
  * d/rules: run test suite on build (LP: #586632)
 -- Clint Byrum <email address hidden> Thu, 09 Feb 2012 15:35:15 -0800

[Rolf Leggewie (r0lf)]

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".