error when using token get from application credential authentication

[Bo Tran]

Currently, from queens version was supported Application Credentials.
But, when I using a token was created from application credentials authentication to create a session after using this session to create a python client, I was get an error log like:

2020-05-13 18:16:50.418018 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi [req-21d4d259-58d1-4e22-997d-ad223bce46a1 393bd2ca957b464e9239ed6216ba964b 29ac25437aa04afa97e53f12a6d7567a - default defau
lt] 'application_credential': KeyError: 'application_credential'
2020-05-13 18:16:50.418053 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi Traceback (most recent call last):
2020-05-13 18:16:50.418060 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 226, in __call__
2020-05-13 18:16:50.418065 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi result = method(req, **params)
2020-05-13 18:16:50.418069 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi File "/usr/lib/python2.7/dist-packages/keystone/auth/controllers.py", line 135, in authenticate_for_token
2020-05-13 18:16:50.418074 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi app_cred_id = token_auth['application_credential']['id']
2020-05-13 18:16:50.418078 2020-05-13 18:16:50.416 34 ERROR keystone.common.wsgi KeyError: 'application_credential'

Steps perform:
1. Create a application credentials with command:
openstack application credential create openstack-ac --secret openstack

2. I using ipython to create session, python client:
from keystoneauth1.identity import v3
from keystoneauth1.session import Session

auth = v3.ApplicationCredential(
auth_url="http://127.0.0.1:5000/v3",
application_credential_id="ca5936c4382c4e8abf4be7ed7b0837c5",
application_credential_secret="openstack")

session = Session(auth=auth)

3. I using above session to get a new token and make a new auth:
auth = v3.Token(
auth_url="http://127.0.0.1:5000/v3",
token=session.get_token(),
project_name="admin",
project_domain_name="Default")

#make a new session
session = Session(auth=auth)

4. Make a keystone client using above session:
keystone = keystone_client.Client(session=session)

# Use keystone client to list projects
keystone.projects.list()

Is this a bug?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/727808

[Colleen Murphy (krinkle)]

You do not need to create a new token or a new session. The application credential session should be used for your keystone client session.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/727819

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/727820

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/727823

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/727824

[Bo Tran (ministry.nd)]

I trying describing about my case, in my company, a token was get from application credentials or from this API: https://docs.openstack.org/api-ref/identity/v3/index.html?expanded=authenticating-with-an-application-credential-detail#authenticating-with-an-application-credential

will bring use by others client. This clients will using that token to make a new session and auth and was get error log like about

[Colleen Murphy (krinkle)]

You should not use a token created from an application credential to create another token. You should simply use the application credential to create the new token. What's happening is keystone is looking up the original token and appending 'application_credential' to the method names because the original token was created with that method. It then expects to find the credential information in the request body and can't, because it's a malformed request containing only the token information. The way forward is not to allow a malformed request - which would override our efforts to ensure application credentials are restricted to the limited scope they were created or - but instead to handle the bad request gracefully and return either a 400 or 401 client error.

[Bo Tran (ministry.nd)]

But I need a token create from application to process for other services. Because my client not provide information of account, and they want some actions do by automation, ...

[Lance Bragstad (lbragstad)]

We discussed this in the keystone team meeting today and we agreed on fixing the 500. Keystone should return a 400 indicating a malformed request.

Bo, your application can use an application credential directly, depending on the library it's using to interact with keystone. You shouldn't need to generate an application credential, authenticate, and then exchange tokens for your application to consume it or interact with other OpenStack APIs.

Can you clarify your use case in more detail?

[Bo Tran (ministry.nd)]

Yep, why we don't improve this?
My use case that is: my applications communicate with each other by a token. But my clients required some auto actions and that actions required client's auth info and they don't want give it to me. I was read Keystone API Document and found application credential.

[Sean Matheny (smat924)]

Just to chime in with our use case as well:
We have many scripts that currently employ admin robot service accounts. We want these to instead use application credentials, for all the reasons. These scripts run openstack client commands in cron jobs frequently (e.g. adding specialised roles or images to projects based on LDAP group membership). Issuing a new token each time one of these scripts run is wasteful and creates additional load on our memcache, thus reusing tokens until they expire was our preferred route.

Of course, we should be using native python libraries that (IIRC) have this functionality built in, and/or deploying other solutions (Mistral, perhaps?), but in the short term it's just not on the roadmap.

So clearly our use case here is one of convenience; if this contravenes the architecture of application credentials as you described above, we understand.

[Sean Matheny (smat924)]

Ah, sorry, I just read Kristi's reply in my (duplicate) bug report after posting the above:
https://bugs.launchpad.net/keystone/+bug/1886607

"What the the openstackclient is trying to do, is authenticate using the token to fetch another token. This appears to break for tokens that have been fetched using application credentials (and would be of limited usability anyway)

You can avoid the reathentication step of the openstackclient by specifying --os-endpoint to directly provide the url of the service you're trying to issue commands to.

For example:
unset OS_AUTH_TYPE
openstack --os-token $TOKEN --os-endpoint http://localhost/identity/v3 project list --verbose"

Maybe I've misunderstood-- if I generate a token using application credentials and then re-use that token, does the openstack client use that token to generate *another* token in the background? If this is the case, then our use case described above isn't doing anything productive and we need to re-work.

[Sean Matheny (smat924)]

I can confirm Kristi's workaround works for me here:

openstack --os-token $TOKEN --os-endpoint http://localhost/identity/v3 project list --verbose

So I'm clear, does this command only use the previously issued $TOKEN (since it skips the re-auth) or does it also generate a new, second token?

[Bo Tran (ministry.nd)]

Yeah, as I said in my commit, I have a question. That is, we have API: https://docs.openstack.org/api-ref/identity/v3/?expanded=authenticating-with-an-application-credential-detail to get a new token from application credentials. Other side, if we have a token from application credential and API: https://docs.openstack.org/api-ref/identity/v3/#token-authentication-with-scoped-authorization to get token from other token. Hence why we can't to do this?

[Kristi Nikolla (knikolla)]

Sean,

The reason for the reauthentication step, is so that the openstack client can receive the service catalog. This is returned to the user after authenticating, and contains the URLs of all the endpoints of all the services, ex. keystone, nova, cinder, etc.

If you provide just a token by itself, it doesn't contain the service catalog, and the client doens't know where to send the next API call. Hence the client uses the token to reauthenticate and use the service catalog and token from the reauthentication.

If you provide the token and the endpoint URL, the openstack client doesn't need to fetch the service catalog and uses that token to talk directly to the endpoint provided.

There currently isn't an API accessible by users/clients to fetch the service catalog by providing a token, except for token authentication and validation. This may be an area for improvement in keystone and clients.

[Kristi Nikolla (knikolla)]

Bo,

If I understand correctly, your code example is simulating a service receiving a request with a token, and that is why you can't use application credentials directly.

It should be possible to instantiate a client in python directly with a token and endpoint, though I haven't tried it.

[Sean Matheny (smat924)]

Kristi, thanks for your excellent explanation... makes perfect sense now.

[Bo Tran (ministry.nd)]

Kristi,
Yep, you are correct.
But my systems was build long ago and I can't break all to build new systems :D
Using application credentials to get token is my idea for new features. :((

[Bo Tran (ministry.nd)]

My applications communicate by application credential token. My issue is token of application credential can't re-use

[Kristi Nikolla (knikolla)]

I lied, there appears to be a /v3/auth/catalog API call to fetch the catalog for a token without reauthenticating https://docs.openstack.org/api-ref/identity/v3/index.html?expanded=get-service-catalog-detail#get-service-catalog

I'm surprised I didn't know about it before. I will investigate how we can update the libraries to make use of that more seamlessly, rather than reauthenticate a token for a token.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bo Tran (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/727823

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bo Tran (<email address hidden>) on branch: stable/queens
Review: https://review.opendev.org/727819

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bo Tran (<email address hidden>) on branch: stable/stein
Review: https://review.opendev.org/727824

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bo Tran (<email address hidden>) on branch: stable/rocky
Review: https://review.opendev.org/727820

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Gage Hugo <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/keystone/+/727808
Reason: Abandoning since there hasn't been any recent activity, if anyone wants to continue this work, please feel free to restore this or create a new change.