Keystone Redis Caching

[Behzad]

As trying to integrate the Keystone with Redis as the caching layer (the procedures are working fine with the Memcached), the following error is being shown:
The information:
OS = Ubuntu 20.04
Openstack repo: Ubuntu cloud (Wallaby edition)
keystone version 19.0.0

When i issue the "openstack endpoint list --service identity" command, and the output shows the "Internal Server Error (HTTP 500)".
====================================
The following error is the error shown in the /var/log/apache2/keystone.log:
here is the link for https://paste.ubuntu.com/p/CmrgZ2JrkX/
the following is the content of the link:

2021-05-01 16:37:17.350115 mod_wsgi (pid=241861): Exception occurred processing WSGI script '/usr/bin/keystone-wsgi-public'.
2021-05-01 16:37:17.358549 Traceback (most recent call last):
2021-05-01 16:37:17.359016 File "/usr/lib/python3/dist-packages/flask/app.py", line 2463, in __call__
2021-05-01 16:37:17.359044 return self.wsgi_app(environ, start_response)
2021-05-01 16:37:17.359072 File "/usr/lib/python3/dist-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__
2021-05-01 16:37:17.359083 return self.app(environ, start_response)
2021-05-01 16:37:17.359104 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__
2021-05-01 16:37:17.359114 resp = self.call_func(req, *args, **kw)
2021-05-01 16:37:17.359134 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func
2021-05-01 16:37:17.359144 return self.func(req, *args, **kwargs)
2021-05-01 16:37:17.359165 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__
2021-05-01 16:37:17.359174 response = req.get_response(self.application)
2021-05-01 16:37:17.359195 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send
2021-05-01 16:37:17.359204 status, headers, app_iter = self.call_application(
2021-05-01 16:37:17.359225 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application
2021-05-01 16:37:17.359239 app_iter = application(self.environ, start_response)
2021-05-01 16:37:17.359260 File "/usr/lib/python3/dist-packages/webob/dec.py", line 143, in __call__
2021-05-01 16:37:17.359271 return resp(environ, start_response)
2021-05-01 16:37:17.359293 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__
2021-05-01 16:37:17.359304 resp = self.call_func(req, *args, **kw)
2021-05-01 16:37:17.359325 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func
2021-05-01 16:37:17.359339 return self.func(req, *args, **kwargs)
2021-05-01 16:37:17.359363 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__
2021-05-01 16:37:17.359373 response = req.get_response(self.application)
2021-05-01 16:37:17.359395 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send
2021-05-01 16:37:17.359406 status, headers, app_iter = self.call_application(
2021-05-01 16:37:17.359429 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application
2021-05-01 16:37:17.359439 app_iter = application(self.environ, start_response)
2021-05-01 16:37:17.359460 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__
2021-05-01 16:37:17.359470 resp = self.call_func(req, *args, **kw)
2021-05-01 16:37:17.359492 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func
2021-05-01 16:37:17.359502 return self.func(req, *args, **kwargs)
2021-05-01 16:37:17.359523 File "/usr/lib/python3/dist-packages/osprofiler/web.py", line 112, in __call__
2021-05-01 16:37:17.359534 return request.get_response(self.application)
2021-05-01 16:37:17.359579 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send
2021-05-01 16:37:17.359589 status, headers, app_iter = self.call_application(
2021-05-01 16:37:17.359611 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application
2021-05-01 16:37:17.359625 app_iter = application(self.environ, start_response)
2021-05-01 16:37:17.359648 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__
2021-05-01 16:37:17.359658 resp = self.call_func(req, *args, **kw)
2021-05-01 16:37:17.359679 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func
2021-05-01 16:37:17.359688 return self.func(req, *args, **kwargs)
2021-05-01 16:37:17.359709 File "/usr/lib/python3/dist-packages/oslo_middleware/request_id.py", line 58, in __call__
2021-05-01 16:37:17.359722 response = req.get_response(self.application)
2021-05-01 16:37:17.359745 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send
2021-05-01 16:37:17.359754 status, headers, app_iter = self.call_application(
2021-05-01 16:37:17.359776 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application
2021-05-01 16:37:17.359785 app_iter = application(self.environ, start_response)
2021-05-01 16:37:17.359806 File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__
2021-05-01 16:37:17.359817 return self.app(environ, start_response)
2021-05-01 16:37:17.359839 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__
2021-05-01 16:37:17.359849 resp = self.call_func(req, *args, **kw)
2021-05-01 16:37:17.359871 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func
2021-05-01 16:37:17.359880 return self.func(req, *args, **kwargs)
2021-05-01 16:37:17.359902 File "/usr/lib/python3/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__
2021-05-01 16:37:17.359912 response = req.get_response(self._app)
2021-05-01 16:37:17.359933 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send
2021-05-01 16:37:17.359942 status, headers, app_iter = self.call_application(
2021-05-01 16:37:17.359963 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application
2021-05-01 16:37:17.359973 app_iter = application(self.environ, start_response)
2021-05-01 16:37:17.359994 File "/usr/lib/python3/dist-packages/werkzeug/middleware/dispatcher.py", line 66, in __call__
2021-05-01 16:37:17.360004 return app(environ, start_response)
2021-05-01 16:37:17.360026 File "/usr/lib/python3/dist-packages/flask/app.py", line 2449, in wsgi_app
2021-05-01 16:37:17.360036 response = self.handle_exception(e)
2021-05-01 16:37:17.360057 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360066 return original_handler(e)
2021-05-01 16:37:17.360087 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360097 return original_handler(e)
2021-05-01 16:37:17.360118 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360127 return original_handler(e)
2021-05-01 16:37:17.360148 [Previous line repeated 27 more times]
2021-05-01 16:37:17.360171 File "/usr/lib/python3/dist-packages/flask/app.py", line 1866, in handle_exception
2021-05-01 16:37:17.360182 reraise(exc_type, exc_value, tb)
2021-05-01 16:37:17.360203 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise
2021-05-01 16:37:17.360213 raise value.with_traceback(tb)
2021-05-01 16:37:17.360233 File "/usr/lib/python3/dist-packages/flask/app.py", line 2446, in wsgi_app
2021-05-01 16:37:17.360243 response = self.full_dispatch_request()
2021-05-01 16:37:17.360264 File "/usr/lib/python3/dist-packages/flask/app.py", line 1951, in full_dispatch_request
2021-05-01 16:37:17.360284 rv = self.handle_user_exception(e)
2021-05-01 16:37:17.360308 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360317 return original_handler(e)
2021-05-01 16:37:17.360338 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360352 return original_handler(e)
2021-05-01 16:37:17.360375 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router
2021-05-01 16:37:17.360385 return original_handler(e)
2021-05-01 16:37:17.360406 [Previous line repeated 27 more times]
2021-05-01 16:37:17.360429 File "/usr/lib/python3/dist-packages/flask/app.py", line 1820, in handle_user_exception
2021-05-01 16:37:17.360440 reraise(exc_type, exc_value, tb)
2021-05-01 16:37:17.360461 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise
2021-05-01 16:37:17.360471 raise value.with_traceback(tb)
2021-05-01 16:37:17.360493 File "/usr/lib/python3/dist-packages/flask/app.py", line 1949, in full_dispatch_request
2021-05-01 16:37:17.360502 rv = self.dispatch_request()
2021-05-01 16:37:17.360523 File "/usr/lib/python3/dist-packages/flask/app.py", line 1935, in dispatch_request
2021-05-01 16:37:17.360533 return self.view_functions[rule.endpoint](**req.view_args)
2021-05-01 16:37:17.360554 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 468, in wrapper
2021-05-01 16:37:17.360565 resp = resource(*args, **kwargs)
2021-05-01 16:37:17.360586 File "/usr/lib/python3/dist-packages/flask/views.py", line 89, in view
2021-05-01 16:37:17.360596 return self.dispatch_request(*args, **kwargs)
2021-05-01 16:37:17.360617 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request
2021-05-01 16:37:17.360627 resp = meth(*args, **kwargs)
2021-05-01 16:37:17.360648 File "/usr/lib/python3/dist-packages/keystone/server/flask/common.py", line 1064, in wrapper
2021-05-01 16:37:17.360659 return f(*args, **kwargs)
2021-05-01 16:37:17.360680 File "/usr/lib/python3/dist-packages/keystone/api/auth.py", line 315, in post
2021-05-01 16:37:17.360690 token = authentication.authenticate_for_token(auth_data)
2021-05-01 16:37:17.360711 File "/usr/lib/python3/dist-packages/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token
2021-05-01 16:37:17.360721 auth_info = core.AuthInfo.create(auth=auth)
2021-05-01 16:37:17.360743 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 126, in create
2021-05-01 16:37:17.360752 auth_info._validate_and_normalize_auth_data(scope_only)
2021-05-01 16:37:17.360773 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data
2021-05-01 16:37:17.360787 self._validate_and_normalize_scope_data()
2021-05-01 16:37:17.360810 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data
2021-05-01 16:37:17.360820 project_ref = self._lookup_project(self.auth['scope']['project'])
2021-05-01 16:37:17.360841 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 196, in _lookup_project
2021-05-01 16:37:17.360851 domain_ref = self._lookup_domain(project_info['domain'])
2021-05-01 16:37:17.360873 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 171, in _lookup_domain
2021-05-01 16:37:17.360882 domain_ref = PROVIDERS.resource_api.get_domain_by_name(
2021-05-01 16:37:17.360903 File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
2021-05-01 16:37:17.360914 __ret_val = __f(*args, **kwargs)
2021-05-01 16:37:17.360935 File "<decorator-gen-28>", line 2, in get_domain_by_name
2021-05-01 16:37:17.360958 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func
2021-05-01 16:37:17.360969 return self.get_or_create(
2021-05-01 16:37:17.360990 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 974, in get_or_create
2021-05-01 16:37:17.361055 key = self.key_mangler(key)
2021-05-01 16:37:17.361083 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 86, in key_mangler
2021-05-01 16:37:17.361095 key = '%s:%s' % (key, invalidation_manager.region_id)
2021-05-01 16:37:17.361117 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 43, in region_id
2021-05-01 16:37:17.361126 return self._invalidation_region.get_or_create(
2021-05-01 16:37:17.361148 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create
2021-05-01 16:37:17.361158 with Lock(
2021-05-01 16:37:17.361179 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 185, in __enter__
2021-05-01 16:37:17.361189 return self._enter()
2021-05-01 16:37:17.361209 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 87, in _enter
2021-05-01 16:37:17.361223 value = value_fn()
2021-05-01 16:37:17.361246 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 977, in get_value
2021-05-01 16:37:17.361256 value = self._get_from_backend(key)
2021-05-01 16:37:17.361277 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1265, in _get_from_backend
2021-05-01 16:37:17.361286 self.backend.get_serialized(key)
2021-05-01 16:37:17.361307 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 288, in get_serialized
2021-05-01 16:37:17.361317 return cast(SerializedReturnType, self.get(key))
2021-05-01 16:37:17.361338 File "/usr/lib/python3/dist-packages/keystone/common/cache/_context_cache.py", line 74, in get
2021-05-01 16:37:17.361348 value = self.proxied.get(key)
2021-05-01 16:37:17.361369 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 245, in get
2021-05-01 16:37:17.361378 raise NotImplementedError()
2021-05-01 16:37:17.361460 NotImplementedError

follwoing is the keystone.conf configuration for both Apache:
========================================================
cat /etc/apache2/sites-enabled/keystone.conf
Listen 5000

<VirtualHost *:5000>
    WSGIScriptAlias / /usr/bin/keystone-wsgi-public
    WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-public
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LimitRequestBody 114688

    <IfVersion >= 2.4>
      ErrorLogFormat "%{cu}t %M"
    </IfVersion>

    ErrorLog /var/log/apache2/keystone.log
    CustomLog /var/log/apache2/keystone_access.log combined

    <Directory /usr/bin>
        <IfVersion >= 2.4>
                Require all granted
        </IfVersion>
        <IfVersion < 2.4>
            Order allow,deny
            Allow from all
        </IfVersion>
    </Directory>
</VirtualHost>

Alias /identity /usr/bin/keystone-wsgi-public
<Location /identity>
    SetHandler wsgi-script
    Options +ExecCGI

    WSGIProcessGroup keystone-public
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
</Location>

====================================
Content of /etc/keystone/keystone.conf

[DEFAULT]
log_dir = /var/log/keystone
default_log_levels = oslo.cache=DEBUG,dogpile.core.dogpile=DEBUG
[application_credential]
[assignment]
[auth]
[cache]
debug_cache_backend = True
backend = dogpile.cache.redis
backend_argument=url:redis://10.10.13.210/6379/0
enabled = true
[catalog]
[cors]
[credential]
[database]
connection = mysql+pymysql://keystone:<email address hidden>/keystone
[domain_config]
[endpoint_filter]
[endpoint_policy]
[eventlet_server]
[extra_headers]
Distribution = Ubuntu
[federation]
[fernet_receipts]
[fernet_tokens]
[healthcheck]
[identity]
[identity_mapping]
[jwt_tokens]
[ldap]
[memcache]
[oauth1]
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[policy]
[profiler]
[receipt]
[resource]
[revoke]
[role]
[saml]
[security_compliance]
[shadow_users]
[token]
provider = fernet
caching = true
driver = keystone.token.persistence.backends.kvs.Token
[tokenless_auth]
[totp]
[trust]
[unified_limit]
[wsgi]