Keystone is not HA with memcache as token persistence driver

Bug #1436324 reported by Boris Bobrov
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Won't Fix
Low
Unassigned

Bug Description

Keystone becomes extremely slow if one of memcached servers, used as token persistence driver, stops working. This happens because Keystone re-initializes memcache client on every call and memcache client loses information about dead servers and time until they are dead.

To reproduce the issue:
1. Set up two memcached instances on, say, virtual machines
2. Set [token]driver=keystone.token.persistence.backends.memcache_pool.Token
3. Set [memcache]servers=192.168.56.101:11211,192.168.56.102:11211 (change ip to your own)
4. Break down one of the servers. By turning the server off for example
5. Try signing in in Horizon.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/167594

Changed in keystone:
assignee: nobody → Boris Bobrov (bbobrov)
status: New → In Progress
tags: added: kilo-rc-potential
Changed in keystone:
importance: Undecided → Medium
tags: removed: kilo-rc-potential
Revision history for this message
Lin Hua Cheng (lin-hua-cheng) wrote :

the issue is not a high priority, but it seems something we'd like to get out sooner so we can deprecate memcache as token backend in Kilo.

tags: added: kilo-rc-potential
Revision history for this message
Lin Hua Cheng (lin-hua-cheng) wrote :

after chatting with Morgan in IRC, removing the rc tag. Issue is a a fundamental problem with how memcache persistence works and a known long standing issue. Not a critical bug that warrants to be added to RC.

Changed in keystone:
importance: Medium → Low
tags: removed: kilo-rc-potential
Revision history for this message
David Stanek (dstanek) wrote :

memcached is not HA by design. This isn't really something we can fix in Keystone, other than to removed this backend. memcached should really only by used in situations where it is a cache and the application can run even with it shut off.

Changed in keystone:
status: In Progress → Won't Fix
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Boris Bobrov (<email address hidden>) on branch: master
Review: https://review.openstack.org/167594
Reason: since fernet is the new default and it doesn't require persistence, I don't think it's worth working on deprecating memcache specifically.

Boris Bobrov (bbobrov)
Changed in keystone:
assignee: Boris Bobrov (bbobrov) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.