Revoke API calls non-existant method in revoke map syncronize

Bug #1289935 reported by Morgan Fainberg on 2014-03-09
38
This bug affects 5 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Critical
Adam Young
keystone (Ubuntu)
Critical
Corey Bryant
Trusty
Critical
Corey Bryant

Bug Description

The "revoke_api" calls a non-existent method on the revoke tree object during the synchronize method. This results in a non-recoverable error in checking validity of a token if there are expired revocation events.

Code block in question:

http://git.openstack.org/cgit/openstack/keystone/tree/keystone/contrib/revoke/core.py?id=a240705b07b852616e39a2b93253f0a9a09a3ef9#n79

        with self._store.get_lock(_TREE_KEY):
            for e in self._current_events:
                if e.revoked_at < cutoff:
                    self.revoke_map.remove(e)
                    self._current_events.remove(e)
                else:
                    break

The code should call self.revoke_map.remove_event(e) not self.revoke_map.remove(e).

Example traceback:

2014-03-08 20:20:59.338 TRACE keystone.common.wsgi TypeError: object of type 'NoneType' has no len()
2014-03-08 20:20:59.338 TRACE keystone.common.wsgi
2014-03-08 20:20:59.340 INFO eventlet.wsgi.server [-] 172.16.28.1 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 400 239 0.004711
2014-03-08 20:20:59.351 DEBUG keystone.middleware.core [-] Auth token not in the request header. Will not build auth context. from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:253
2014-03-08 20:20:59.352 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.353 ERROR keystone.common.wsgi [-] object of type 'NoneType' has no len()
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi Traceback (most recent call last):
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/wsgi.py", line 205, in __call__
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi result = method(context, **params)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/openstack/common/versionutils.py", line 102, in wrapped
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi return func(*args, **kwargs)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 97, in authenticate
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi context, auth)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 255, in _authenticate_local
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi if len(username) > CONF.max_param_size:
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi TypeError: object of type 'NoneType' has no len()
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi
2014-03-08 20:20:59.355 INFO eventlet.wsgi.server [-] 172.16.28.1 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 400 239 0.004078
2014-03-08 20:20:59.385 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.386 INFO eventlet.wsgi.server [-] 172.16.28.100 - - [08/Mar/2014 20:20:59] "GET / HTTP/1.1" 300 1103 0.001378
2014-03-08 20:20:59.401 DEBUG keystone.middleware.core [-] Auth token not in the request header. Will not build auth context. from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:253
2014-03-08 20:20:59.403 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.412 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-requests/1.2.3 CPython/2.7.5+ Linux/3.11.0-12-generic', 'address': '172.16.28.100'}, 'id': 'openstack:b0d57b38-6f65-43aa-b0ef-b807db297e5b', 'name': u'5b55216e7b1742978dca4ce4f721a6d3'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:006ecd17-f59d-4bc4-9fb5-cde076e7607c'}, 'observer': {'typeURI': 'service/security', 'id': 'openstack:5b7eecb3-de9b-486c-9683-11d50d965cf8'}, 'eventType': 'activity', 'eventTime': '2014-03-08T19:20:59.412018+0000', 'action': 'authenticate', 'outcome': 'pending', 'id': 'openstack:41e1caa6-4e8d-47f9-8a87-3e5d23c2e22d'} from (pid=14327) _send_audit_notification /opt/stack/keystone/keystone/notifications.py:289
2014-03-08 20:20:59.447 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-requests/1.2.3 CPython/2.7.5+ Linux/3.11.0-12-generic', 'address': '172.16.28.100'}, 'id': 'openstack:b0d57b38-6f65-43aa-b0ef-b807db297e5b', 'name': u'5b55216e7b1742978dca4ce4f721a6d3'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:86370275-85d2-4243-bb59-d6c9d93d329c'}, 'observer': {'typeURI': 'service/security', 'id': 'openstack:ea11d624-61f7-4dbb-a6af-0317dfeb5770'}, 'eventType': 'activity', 'eventTime': '2014-03-08T19:20:59.446496+0000', 'action': 'authenticate', 'outcome': 'success', 'id': 'openstack:5874fedc-6212-4367-a842-6ac1ac51015c'} from (pid=14327) _send_audit_notification /opt/stack/keystone/keystone/notifications.py:289
2014-03-08 20:20:59.538 INFO eventlet.wsgi.server [-] 172.16.28.100 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 200 9140 0.136870
2014-03-08 20:20:59.543 DEBUG keystone.middleware.core [-] RBAC: auth_context: {'project_id': u'8d9ffd4e5688425caea13f16473c3e64', 'user_id': u'5b55216e7b1742978dca4ce4f721a6d3', 'roles': [u'_member_', u'admin']} from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:263
2014-03-08 20:20:59.545 DEBUG keystone.common.wsgi [-] arg_dict: {'token_id': u'd5f1e4259de4c4449dc8b4638e6ec0f7'} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.545 DEBUG keystone.common.controller [-] RBAC: Authorizing identity:validate_token(token_id=d5f1e4259de4c4449dc8b4638e6ec0f7) from (pid=14327) _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:40
2014-03-08 20:20:59.546 DEBUG keystone.common.controller [-] RBAC: using auth context from the request environment from (pid=14327) _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:45
2014-03-08 20:20:59.546 DEBUG keystone.policy.backends.rules [-] enforce identity:validate_token: {'project_id': u'8d9ffd4e5688425caea13f16473c3e64', 'user_id': u'5b55216e7b1742978dca4ce4f721a6d3', 'roles': [u'_member_', u'admin']} from (pid=14327) enforce /opt/stack/keystone/keystone/policy/backends/rules.py:100
2014-03-08 20:20:59.547 DEBUG keystone.openstack.common.policy [-] Rule identity:validate_token will be now enforced from (pid=14327) enforce /opt/stack/keystone/keystone/openstack/common/policy.py:258
2014-03-08 20:20:59.548 DEBUG keystone.common.controller [-] RBAC: Authorization granted from (pid=14327) inner /opt/stack/keystone/keystone/common/controller.py:137
2014-03-08 20:20:59.551 DEBUG keystone.common.kvs.core [-] KVS lock acquired for: os-revoke-tree from (pid=14327) acquire /opt/stack/keystone/keystone/common/kvs/core.py:375
2014-03-08 20:20:59.552 DEBUG keystone.common.kvs.core [-] KVS lock released for: os-revoke-tree from (pid=14327) release /opt/stack/keystone/keystone/common/kvs/core.py:394
2014-03-08 20:20:59.553 ERROR keystone.common.wsgi [-] 'RevokeTree' object has no attribute 'remove'
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi Traceback (most recent call last):
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/wsgi.py", line 205, in __call__
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi result = method(context, **params)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/openstack/common/versionutils.py", line 102, in wrapped
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return func(*args, **kwargs)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/controller.py", line 138, in inner
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return f(self, context, *args, **kwargs)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 411, in validate_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return self.token_provider_api.validate_v2_token(token_id, belongs_to)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/provider.py", line 137, in validate_v2_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.check_revocation_v2(token)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/provider.py", line 130, in check_revocation_v2
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.revoke_api.check_token(token_values)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/contrib/revoke/core.py", line 190, in check_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self._cache.synchronize_revoke_map(self.driver)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/contrib/revoke/core.py", line 79, in synchronize_revoke_map
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.revoke_map.remove(e)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi AttributeError: 'RevokeTree' object has no attribute 'remove'
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi

Related branches

Changed in keystone:
importance: Undecided → Critical
status: New → Triaged
assignee: nobody → Morgan Fainberg (mdrnstm)
milestone: none → icehouse-rc1

Fix proposed to branch: master
Review: https://review.openstack.org/79174

Changed in keystone:
status: Triaged → In Progress
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Adam Young (ayoung)

Reviewed: https://review.openstack.org/79174
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3035a6b394ba4b460d9ea18409fa0cb87c86d38d
Submitter: Jenkins
Branch: master

commit 3035a6b394ba4b460d9ea18409fa0cb87c86d38d
Author: Morgan Fainberg <email address hidden>
Date: Sat Mar 8 21:57:51 2014 -0800

    Call an existing method in sync cache for revoke events

    The cache used for synchronizing the revocation tree across
    green threads had an issue where it was calling a non-existant
    method ``remove`` instead of ``remove_event``. The correct method
    is now being called and an expanded test to exercise the synchronize
    method has been added.

    Change-Id: I3fe47fa51f88aab89480831b2d95746319f82ceb
    Closes-Bug: 1289935

Changed in keystone:
status: In Progress → Fix Committed
Chris J Arges (arges) wrote :

This also affects the keystone version in Trusty.

Changed in keystone (Ubuntu):
importance: Undecided → Critical
Changed in keystone (Ubuntu):
status: New → Confirmed
assignee: nobody → Corey Bryant (corey.bryant)
James Page (james-page) on 2014-03-12
Changed in keystone (Ubuntu Trusty):
status: Confirmed → In Progress
James Page (james-page) on 2014-03-13
Changed in keystone (Ubuntu Trusty):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package keystone - 1:2014.1~b3-0ubuntu3

---------------
keystone (1:2014.1~b3-0ubuntu3) trusty; urgency=medium

  * d/p/revoke-api.patch: Add upstream patch to resolve critical issue with
    token revocation (LP: #1289935).
  * d/keystone.postinst: Ensure db_sync is only run when the default sqlite
    connection is configured (LP: #1290423).
 -- Corey Bryant <email address hidden> Wed, 12 Mar 2014 23:20:05 -0500

Changed in keystone (Ubuntu Trusty):
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-03-26
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in keystone:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers