Incorrect MAX_NUMBER_OF_SLAB_CLASSES results in crash for memcached > 1.4.23

Bug #1534062 reported by Matthijs
32
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Moodle
New
Undecided
Unassigned
libmemcached
High
Brian Aker
php-memcached
Fix Released
Unknown

Bug Description

We recently upgraded our memcached server from 1.4.22 to 1.4.25 With this upgrade our homemade script that displays a (sorted) output of all memcache keys did no longer work. If relevant: it is a PHP script using the following call https://secure.php.net/manual/en/memcached.getallkeys.php

Some info:
libmemcached 1.0.4
memcached 1.4.25
php 5.6.17
centos 5

We debugged the issue and found that the "MAX_NUMBER_OF_SLAB_CLASSES" was changed in memcached 1.4.23

So, what goes wrong:
In the file dump.cc (https://bazaar.launchpad.net/~tangent-trunk/libmemcached/1.0/view/head:/libmemcached/dump.cc)
on line 50 is defined that there are 200 slab classes

However this is no longer the case:
In this commit https://github.com/memcached/memcached/commit/9bce42f27c88a88f94c5e5345ced205a0ab36e89 the MAX_NUMBER_OF_SLAB_CLASSES is changed to 63

So when memcached_dump is executed the memcached server will return an error (Illegal slab ID if I'm not mistaking) and this will output the error:

"libmemcached/dump.cc:125: memcached_return_t ascii_dump(memcached_st*, memcached_return_t (**)(const memcached_st*, const char*, size_t, void*), void*, uint32_t): Assertion `response_rc == MEMCACHED_SUCCESS' failed."

Please not that in this commit https://github.com/memcached/memcached/commit/a2fc8e93da7b3ad96505756c84114b34a3644d58 the MAX_NUMBER_OF_SLAB_CLASSES is changed again to "63+1"

So:
It seems that fetching more slabs than there are available is not a good idea :)

To me it seems there are some solutions:
- Simply change the MAX_NUMBER_OF_SLAB_CLASSES to 64 however this makes libmemcached incompatible with memcached < 1.4.22
- Detect the (specific) error and return everything already fetched

How to reproduce:
- Use libmemcached 1.0.4 (or newer, as far as I can see it is an issue)
- Use memcached 1.4.23 or newer
- Execute the 'memcached_dump' call and see that it fails
- And/Or telnet to ip.of.memcached.server 11211 and call 'stats cachedump 65'

If more info is needed please let me know.

Related branches

Revision history for this message
Andrew Nicols (dobedobedoh) wrote :

I'm seeing this too and it's causing a range of breakages as a result.
In our case we use getAllKeys to filter keys for a targetted purge. As a result, our cache is not purged leading to unreliable behaviour.

As an alternative to the above suggestions, calling a `stats slabs` or `stats items` command returns a list of the stats for each slab.
It should be reasonably easy to loop over the results and calls the relevant `stats cachedump %u 0 0\r\n` on just the slabs returned and would result in one single call to fetch the list of slabs, vs 63/256 calls to return a list of empty slabs.

Revision history for this message
Matthijs (matthijs-1) wrote :

Hi Andrew, yes that is sort of the same usecase we have. And yes this problem is still not fixed (as you can see I never even got a reply). So unfortunately for now we'll stick to version 1.4.22

I'm still not sure if there is an "get slabs amount" call but your solution should work :)

Hoping the devs can at least reply to this issue!

Changed in php-memcached:
status: Unknown → New
Revision history for this message
nadavkav (nadavkav) wrote :

I can confirm we are also experiencing this major issue in our servers.

Changed in php-memcached:
status: New → Fix Released
Brian Aker (brianaker)
Changed in libmemcached:
status: New → In Progress
milestone: none → 1.0.19
importance: Undecided → High
assignee: nobody → Brian Aker (brianaker)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.