"ValueError: Expecting a string None" with keystoneauth 3.0.1

[Thomas Bechtold]

When running the tests with "tox -epy27" together with the newly released keystoneauth1 3.0.1, the following errors occur:

[ 143s] ${PYTHON:-python} -m subunit.run discover -t ./ ./ --load-list /tmp/tmpsLZZjS
[ 143s] ======================================================================
[ 143s] FAIL: troveclient.tests.test_shell.ShellTest.test_datastore_version_list
[ 143s] tags: worker-1
[ 143s] ----------------------------------------------------------------------
[ 143s] Traceback (most recent call last):
[ 143s] File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
[ 143s] return func(*args, **keywargs)
[ 143s] File "/usr/lib/python2.7/site-packages/requests_mock/mocker.py", line 219, in inner
[ 143s] return func(*args, **kwargs)
[ 143s] File "troveclient/tests/test_shell.py", line 182, in test_datastore_version_list
[ 143s] stdout, stderr = self.shell('datastore-version-list XXX')
[ 143s] File "troveclient/tests/test_shell.py", line 107, in shell
[ 143s] _shell.main(argstr.split())
[ 143s] File "troveclient/shell.py", line 531, in main
[ 143s] auth=keystone_auth)
[ 143s] File "troveclient/client.py", line 507, in Client
[ 143s] return client_class(*args, **kwargs)
[ 143s] File "troveclient/v1/client.py", line 128, in __init__
[ 143s] **kwargs)
[ 143s] File "troveclient/client.py", line 472, in _construct_http_client
[ 143s] **kwargs)
[ 143s] File "troveclient/client.py", line 428, in __init__
[ 143s] endpoint = self.get_endpoint()
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 226, in get_endpoint
[ 143s] return self.session.get_endpoint(auth or self.auth, **kwargs)
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 947, in get_endpoint
[ 143s] return auth.get_endpoint(self, **kwargs)
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 378, in get_endpoint
[ 143s] allow_version_hack=allow_version_hack, **kwargs)
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 270, in get_endpoint_data
[ 143s] project_id = self.get_project_id(session)
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 385, in get_project_id
[ 143s] return self.get_access(session).project_id
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
[ 143s] if self._needs_reauthenticate():
[ 143s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 106, in _needs_reauthenticate
[ 146s] if self.auth_ref.will_expire_soon(self.MIN_TOKEN_LIFE_SECONDS):
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 94, in will_expire_soon
[ 146s] norm_expires = utils.normalize_time(self.expires)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 57, in inner
[ 146s] return f(self)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 429, in expires
[ 146s] return utils.parse_isotime(self._token.get('expires'))
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/_utils.py", line 41, in parse_isotime
[ 146s] raise ValueError(six.text_type(e))
[ 146s] ValueError: Expecting a string None
[ 146s] ======================================================================
[ 146s] FAIL: troveclient.tests.test_shell.ShellTest.test_get_datastore_list
[ 146s] tags: worker-7
[ 146s] ----------------------------------------------------------------------
[ 146s] Traceback (most recent call last):
[ 146s] File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
[ 146s] return func(*args, **keywargs)
[ 146s] File "/usr/lib/python2.7/site-packages/requests_mock/mocker.py", line 219, in inner
[ 146s] return func(*args, **kwargs)
[ 146s] File "troveclient/tests/test_shell.py", line 203, in test_get_datastore_list
[ 146s] stdout, stderr = self.shell('datastore-list')
[ 146s] File "troveclient/tests/test_shell.py", line 107, in shell
[ 146s] _shell.main(argstr.split())
[ 146s] File "troveclient/shell.py", line 531, in main
[ 146s] auth=keystone_auth)
[ 146s] File "troveclient/client.py", line 507, in Client
[ 146s] return client_class(*args, **kwargs)
[ 146s] File "troveclient/v1/client.py", line 128, in __init__
[ 146s] **kwargs)
[ 146s] File "troveclient/client.py", line 472, in _construct_http_client
[ 146s] **kwargs)
[ 146s] File "troveclient/client.py", line 428, in __init__
[ 146s] endpoint = self.get_endpoint()
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 226, in get_endpoint
[ 146s] return self.session.get_endpoint(auth or self.auth, **kwargs)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 947, in get_endpoint
[ 146s] return auth.get_endpoint(self, **kwargs)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 378, in get_endpoint
[ 146s] allow_version_hack=allow_version_hack, **kwargs)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 270, in get_endpoint_data
[ 146s] project_id = self.get_project_id(session)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 385, in get_project_id
[ 146s] return self.get_access(session).project_id
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
[ 146s] if self._needs_reauthenticate():
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 106, in _needs_reauthenticate
[ 146s] if self.auth_ref.will_expire_soon(self.MIN_TOKEN_LIFE_SECONDS):
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 94, in will_expire_soon
[ 146s] norm_expires = utils.normalize_time(self.expires)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 57, in inner
[ 146s] return f(self)
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/access/access.py", line 429, in expires
[ 146s] return utils.parse_isotime(self._token.get('expires'))
[ 146s] File "/usr/lib/python2.7/site-packages/keystoneauth1/_utils.py", line 41, in parse_isotime
[ 146s] raise ValueError(six.text_type(e))
[ 146s] ValueError: Expecting a string None
[ 146s] Ran 415 tests in 3.347s
[ 146s] FAILED (id=0, failures=2)
[ 146s] error: testr failed (1)

This can also be seen at: https://review.openstack.org/#/c/485964/ (global-requirements update which bumps keystoneauth1 to 3.0.1)

[Lance Bragstad (lbragstad)]

I've proposed a patch to python-troveclient to adjust a token mock that was using the wrong expiration attribute for v2.0 tokens. It just happen to be keystoneauth that stumbled across the error because it expects `expires` to be in the token reference. There was a lot of shuffling in the last release of keystoneauth, so a shift in logic, or how keystoneauth is invoked probably triggered this.

https://github.com/openstack/python-troveclient/blob/555ba5ed690027088358288c9b0252d5a58bdc0d/troveclient/tests/test_shell.py#L77

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/487417
Committed: https://git.openstack.org/cgit/openstack/python-troveclient/commit/?id=47ad24a358104d41b915f25cdd5a17d915c6ee75
Submitter: Jenkins
Branch: master

commit 47ad24a358104d41b915f25cdd5a17d915c6ee75
Author: Lance Bragstad <email address hidden>
Date: Wed Jul 26 13:18:47 2017 +0000

    Fix token response mock

    The unit tests mock tokens from keystoneauth. One of the responses
    being returned from a mocked keystoneauth method was returning a v2.0
    token that didn't have the right expiration attribute of a v2.0
    token. This was caught by keystoneauth 3.0.1, which underwent a
    significant refactor and added a bunch of new functionality for
    version discovery. Keystoneauth was expecting `expires` to be
    in the token but instead the mock was using `expires_at`, which isn't
    a valid v2.0 token attribute.

    Closes-Bug: 1706538

    Change-Id: Iea77ef54585d0b3480369e9f5df629e3f46f7e3b

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-troveclient 2.12.0 release.