Fernet TypeError: token must be bytes.

Bug #1431669 reported by Haneef Ali on 2015-03-13
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Dolph Mathews

Bug Description

TypeError: token must be bytes.
(keystone.common.wsgi): 2015-03-13 03:04:16,968 ERROR token must be bytes.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 238, in __call__
    result = method(context, **params)
  File "/usr/local/lib/python2.7/dist-packages/keystone/common/validation/__init__.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystone/contrib/federation/controllers.py", line 328, in create_saml_assertion
    token_data = self.token_provider_api.validate_token(token_id)
  File "/usr/local/lib/python2.7/dist-packages/keystone/token/provider.py", line 196, in validate_token
    token = self._validate_token(unique_id)
  File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1040, in decorate
  File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 651, in get_or_create
    async_creator) as value:
  File "/usr/local/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 158, in __enter__
    return self._enter()
  File "/usr/local/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 98, in _enter
    generated = self._enter_create(createdtime)
  File "/usr/local/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 149, in _enter_create
    created = self.creator()
  File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 619, in gen_value
    created_value = creator()
  File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1036, in creator
    return fn(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/keystone/token/provider.py", line 257, in _validate_token
    return self.driver.validate_v3_token(token_id)
  File "/usr/local/lib/python2.7/dist-packages/keystone/token/providers/fernet/core.py", line 150, in validate_v3_token
  File "/usr/local/lib/python2.7/dist-packages/keystone/token/providers/fernet/token_formatters.py", line 223, in validate_token
    payload = self.unpack(token_string)
  File "/usr/local/lib/python2.7/dist-packages/keystone/token/providers/fernet/token_formatters.py", line 125, in unpack
    decrypted_token = self.crypto.decrypt(token_string)
  File "/usr/local/lib/python2.7/dist-packages/cryptography/fernet.py", line 138, in decrypt
    return f.decrypt(msg, ttl)
  File "/usr/local/lib/python2.7/dist-packages/cryptography/fernet.py", line 75, in decrypt
    raise TypeError("token must be bytes.")
TypeError: token must be bytes.

Haneef Ali (haneef) on 2015-03-13
tags: added: fernet
Haneef Ali (haneef) wrote :

BTW, this is with the master branch code as of 8:00 PM PST 03/12

Haneef Ali (haneef) wrote :

INFO:__main__:{"auth": {"scope": {"service_provider": {"id": "ks_sp1"}}, "identity": {"token": {"id": "gAAAAABVAldqGaZeAQbn37Vh4Gdb2CUfn2UHTCoqWKUTiFD8S0R6wc8SfuzEhk90xrQwDM415hodebk0HjjPH479cj3f6eIh05jR41cSouiYTRyB8yH6TvUQhRk6hei2O5dM6AEwii0uFodJ6zspuF62xyDWISn41_sFwcpv3fvrbgz2-nzQHPU="}, "methods": ["token"]}}}

Dolph Mathews (dolph) wrote :

I reproduced this a bit more simply by rescoping a token from unscoped to project-scoped using v3:

ERROR: test_token_rescoping (test_exercises.TestCase)
Traceback (most recent call last):
  File "/home/dolph/dolph/keystone-deploy/test_exercises.py", line 192, in test_token_rescoping
    auth_url=KEYSTONE_ENDPOINT + 'v3')
  File "/home/dolph/Environments/ansible/local/lib/python2.7/site-packages/keystoneclient/v3/client.py", l
ine 196, in __init__
  File "/home/dolph/Environments/ansible/local/lib/python2.7/site-packages/keystoneclient/utils.py", line
318, in inner
    return func(*args, **kwargs)
  File "/home/dolph/Environments/ansible/local/lib/python2.7/site-packages/keystoneclient/httpclient.py",
line 503, in authenticate [293/1965]
    resp = self.get_raw_token_from_identity_service(**kwargs)
  File "/home/dolph/Environments/ansible/local/lib/python2.7/site-packages/keystoneclient/v3/client.py", l
ine 281, in get_raw_token_from_identity_service
    _('Authorization failed: %s') % e)
AuthorizationFailure: Authorization failed: token must be bytes. (HTTP 400)

The backtrace can be reproduced in isolation with just:

  $ python -c "from cryptography.fernet import Fernet; f = Fernet(Fernet.generate_key()).decrypt(u'asdf')"

I think the cause is that tokens which pass through json.loads() are read in as unicode, and pypi/cryptography simply refuses to operate on them. It should be safe to cast them to bytes()

Changed in keystone:
assignee: nobody → Dolph Mathews (dolph)
importance: Undecided → High
status: New → Triaged
summary: - Create saml assertion doesn't work with fernet token
+ Fernet TypeError: token must be bytes.
Changed in keystone:
status: Triaged → In Progress
Lance Bragstad (lbragstad) wrote :

here is the review addressing this:


Changed in keystone:
milestone: none → kilo-rc1

Reviewed: https://review.openstack.org/165279
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=68a54c2cf1a7210abae9292aede5737203eb733a
Submitter: Jenkins
Branch: master

commit 68a54c2cf1a7210abae9292aede5737203eb733a
Author: Dolph Mathews <email address hidden>
Date: Wed Mar 18 03:18:31 2015 +0000

    URL quote Fernet tokens

    The padding in base64.urlsafe_b64encode()'s output is not actually URL
    safe, so you have to quote the result when it's of variable length.

    In addition, Fernet tokens can always be handled as bytes, despite being
    passed in from json.loads() as Unicode.

    Change-Id: I72dbd4ddc066706f6af6ea2f2bcd5f0a6cb9b30c
    Closes-Bug: 1433372
    Closes-Bug: 1431669

Changed in keystone:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2015-04-07
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2015-04-30
Changed in keystone:
milestone: kilo-rc1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers