Cannot encode revokeTree object when mongo is configured as cache

[Jose Castro Leon]

When you configure mongo as the cache subsystem for keystone, it fails to store the revoke tree in the cache. This is due that the basetransform only expects single types or dictionaries.

    def transform_incoming(self, son, collection):
        """Used while saving data to MongoDB."""
        for (key, value) in list(son.items()):
            if isinstance(value, api.CachedValue):
                son[key] = value.payload # key is 'value' field here
                son['meta'] = value.metadata
            elif isinstance(value, dict): # Make sure we recurse into sub-docs
                son[key] = self.transform_incoming(value, collection)
        return son

In mitaka this code has been sent to oslo_cache.mongo but the issue is still there

As a result there is a 500 error on the response for validating a token.

This is the exception I am getting

2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi [req-9508d2e4-4e4c-4fd1-8daa-bc7c3456a320 - - - - -] Cannot encode object: <keystone.contrib.revoke.model.RevokeTree object at 0x7f5b0f177a90>
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi Traceback (most recent call last):
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 452, in __call__
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi response = self.process_request(request)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/middleware/core.py", line 305, in process_request
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi auth_context = self._build_auth_context(request)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/middleware/core.py", line 193, in _build_auth_context
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi token_data=self.token_provider_api.validate_token(token_id))
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/token/provider.py", line 190, in validate_token
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self._is_valid_token(token)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/token/provider.py", line 296, in _is_valid_token
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self.check_revocation(token)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/token/provider.py", line 230, in check_revocation
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi return self.check_revocation_v3(token)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/token/provider.py", line 223, in check_revocation_v3
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self.revoke_api.check_token(token_values)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/contrib/revoke/core.py", line 226, in check_token
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi if self._get_revoke_tree().is_revoked(token_values):
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1040, in decorate
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi should_cache_fn)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 651, in get_or_create
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi async_creator) as value:
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 158, in __enter__
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi return self._enter()
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 98, in _enter
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi generated = self._enter_create(createdtime)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 149, in _enter_create
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi created = self.creator()
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 624, in gen_value
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self.backend.set(key, value)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/cache/backends/mongo.py", line 186, in set
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self.client.set(key, value)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/cache/backends/mongo.py", line 443, in set
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi **self.meth_kwargs)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1390, in find_and_modify
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi **kwargs)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 388, in command
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi result = self["$cmd"].find_one(command, **extra_opts)
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 596, in find_one
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi for result in self.find(spec_or_id, *args, **kwargs).limit(-1):
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 814, in next
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi if len(self.__data) or self._refresh():
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 763, in _refresh
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi self.__uuid_subtype))
2016-06-23 06:42:13.248 30288 ERROR keystone.common.wsgi InvalidDocument: Cannot encode object: <keystone.contrib.revoke.model.RevokeTree object at 0x7f5b0f177a90>

I am opening this here, because with the change from revoketree to revokeevents, maybe there is something to be checked here as well.

[Dolph Mathews (dolph)]

Moved this to oslo.cache, since keystone.common.cache.backends.mongo:MongoCacheBackend moved to oslo_cache.backends.mongo:MongoCacheBackend. I'm guessing we need to add a third case to that method to attempt to pickle complex objects?

[Dolph Mathews (dolph)]

(I'm assuming that the core issue is that the mongo backend does not have feature parity with other dogpile backends, and so the issue is not with keystone's usage of the backend.)

[Morgan Fainberg (mdrnstm)]

Is anyone using the mongo backend? If not, we should remove it/deprecate it. This seems like a long unused backend if this is still an issue.

[Launchpad Janitor (janitor)]

[Expired for oslo.cache because there has been no activity for 60 days.]