Comment 30 for bug 1832265

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

Ran into a related problem during debugging of dashboard errors ("Unable to retrieve key pairs") with a Rocky cloud & identity federation.

There was no clear indication as to why failures occurred.

https://paste.ubuntu.com/p/v5HXyyWXC2/ (full pdb trace)

At a high level I was getting validation failures for the identity provider (which was enabled in Keystone and was otherwise correct in terms of config) in the /v3/auth/token code path.

I narrowed it down to a validation error due to a type mismatch (bytes vs str):

1) the error occurs in send_notification:

> /usr/lib/python3/dist-packages/keystone/auth/plugins/mapped.py(101)handle_scoped_token()->None
-> send_notification(taxonomy.OUTCOME_SUCCESS)
(Pdb) l
 96 # send off failed authentication notification, raise the exception
 97 # after sending the notification
 98 send_notification(taxonomy.OUTCOME_FAILURE)
 99 raise
100 else:
101 -> send_notification(taxonomy.OUTCOME_SUCCESS)

# ...

2) this is how the validation error looks like:

(Pdb) setattr(self, FED_CRED_KEYNAME_IDENTITY_PROVIDER, identity_provider)
*** ValueError: identity_provider failed validation: <function FederatedCredential.<lambda> at 0x7fa0016ef9d8>

3) the lambda function where the error occurs

 67 class FederatedCredential(Credential):
 68 identity_provider = cadftype.ValidatorDescriptor(
 69 FED_CRED_KEYNAME_IDENTITY_PROVIDER,
 70 -> lambda x: isinstance(x, six.string_types))
 71 user = cadftype.ValidatorDescriptor(
 72 FED_CRED_KEYNAME_USER,
 73 lambda x: isinstance(x, six.string_types))
 74 groups = cadftype.ValidatorDescriptor(
 75 FED_CRED_KEYNAME_GROUPS,

4) type comparison (b'adfs' is the identity provider name):

((Pdb)) x
b'adfs'
((Pdb)) six.string_types
(<class 'str'>,)
((Pdb)) type(x)
<class 'bytes'>

Using a package from James' PPA helped as I am not getting errors in the same code-path anymore.

apt policy keystone
keystone:
  Installed: 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719
  Candidate: 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719
  Version table:
 *** 2:14.1.0-0ubuntu2~ubuntu18.04.1~ppa201906140719 500

When clicking through tabs very fast I encountered a glitch which results in the following error messages being displayed (see the screencast in the attachment):

Error: "Unable to retrieve key pairs"/"Unable to retrieve images"/""Unable to retrieve server groups"
Warning: "Policy check failed"

I tried to set breakpoints in the same place - the same validation error does NOT occur with the patch so this is something else unrelated to py2 vs py3 string handling.