Default keystone token backend in KILO is not HA

[Serge van Ginderachter]

In the kilo release, the default token backend for keystone uses memcached, pointing to a list of memcaches that run in each keystone container.

Given that keystone needs the same set of servers on all its instances, as to make all tokens available to all instances, whilst tokens are only kept on 1 specific memcache instance (based on a hash algorythm) this implies that all memcache instances need to be available at all time.

Memcache does not federate its data, so the loss of an instance means the loss of a token, which yields authenitcation errors both in frontend and backend applications.

This setup is not HA, and should definitely not be the default one proposed. Currently the best quick alternative is to use mysql as a token backend - which was the default in Juno. (set keystone_token_driver: keystone.token.persistence.backends.sql.Token in inventory)

The better solution is to have the fernet token backend, which is an upcoming feature to be implemented in OSAD, and will probably be backported to Kilo.

This issue was discussed with the OSAD team. Depending on how fast this updates lands in Kilo, one should however consider the best option to solve this in the current kilo release ASAP.

[Kevin Carter (kevin-carter)]

in tomorrows meeting, Juno 25, 2015, well be talking about the use of fernet as the default in the Kilo branch for the 11.x.y series. Should fernet not be ratified, we'll look at going back to sql backed token storage which is HA capable at the detriment of performance. If you have time we'd love for your input into the meeting on this topic [ https://wiki.openstack.org/wiki/Meetings/openstack-ansible ].

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: kilo
Review: https://review.openstack.org/196499

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/196499
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=fafcafa4dfff613fd7e591da9bf68aabbbf2553e
Submitter: Jenkins
Branch: kilo

commit fafcafa4dfff613fd7e591da9bf68aabbbf2553e
Author: kevin <email address hidden>
Date: Thu Jun 25 21:15:11 2015 -0500

    Updated default fernet key usage

    This change makes the use of fernet tokens production ready. The changes are
    as follows:
      * Ensures that the keys are rotated on every playbook execution
      * Removes the need to sync keys back to a deployment host when distributing
        them to other keystone hosts.
      * Creates an autonomous key rotation process that can rotate on the following
        intervals [reboot, yearly, annually, monthly, weekly, daily, hourly] to all
        hosts from any keystone fernet host.
      * Fixes the section in `keystone.conf` which was named "fernet_key" instead
        of "fernet_token".

    Change-Id: I50f6a852930728631f5c681a8aa0f1321d7424ac
    Related-Bug: #1463569
    Closes-Bug: #1468256
    (cherry picked from commit df3edca7a6def8869479feb98ea815f0bc7d30a4)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.