Canonical Juju

OpenStack Credentials: support Keystone application credentials

Bug #1834433 reported by Dmitrii Shcherbakov on 2019-06-27

This bug affects 7 people

Affects		Status	Importance	Assigned to	Milestone
	Canonical Juju	Triaged	Wishlist	Unassigned

Bug Description

Keystone has support for application credentials as of Queens (for API, CLI and dashboard support was added in Rocky). It is quite useful for clouds integrated with user directories via LDAP or with identity federation & SSO enabled where it is not possible or meaningful to use credentials of a user in a company-wide user directory.

https://docs.openstack.org/keystone/rocky/user/application_credentials.html
https://developer.openstack.org/api-ref/identity/v3/?expanded=create-application-credential-detail#create-application-credential

Juju does not support this at the time of writing.
https://github.com/juju/juju/blob/2.6.3/provider/openstack/credentials.go#L25-L26

* This feature allows a user to create a credential via Keystone with some of its role assignments delegated to a credential (a "Member" project role is usually delegated);
* Expiration date is possible to configure for a credential but is optional;
* Unrestricted credentials can be optionally created to create new credentials;
* There is no API to change an expiration time of an existing credential if it is set.

Example openrc:

export OS_AUTH_URL=https://keystone.maas:5000/v3
export OS_APPLICATION_CREDENTIAL_SECRET=DEADBEEFtWBOt9KPlWfCnhnBZKQShimC9uu0w-VNetgxOOC5KtY3KTAZXuDa7LPaXxZpo3nbhdbRoaqkULewvg
export OS_REGION_NAME=RegionOne
export OS_APPLICATION_CREDENTIAL_ID=213b49eda51d4c23b92f5f4f87c66fa7
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_TYPE=v3applicationcredential
export OS_INTERFACE=internal
export OS_CACERT=/home/ubuntu/bundles/vault-ca.crt

Example API client usage:

>>> from keystoneauth1.identity import v3
>>> from keystoneauth1 import session
>>> from keystoneclient.v3 import client
>>> auth = v3.token.Token(auth_url="https://keystone.maas:5000/v3", token='gAAAAABdB8ngthtomqC5IvpWMUwEFHHpdN--8ywI4-cmvjG9lYnuJCBaD04BRrvVYIqLuEFfgG5-tV1ySNxuWv-3Qlaf5dqJrFh_NIU8_9vi_c4witJxqYaIFSzvCzLcyldN1m70tMBjq1CXCXbEL7uRlPVSDvSu62wgq72GwroxJUg5Zyx-dHwW_XnyqA51QopLTdi5vYt7', project_id='11183e94d6e344a4b81aae42fa4e716e')
>>> sess = session.Session(auth=auth, verify='/home/ubuntu/bundles/vault-ca.crt')
>>> keystone = client.Client(session=sess)
>>> app_cred = keystone.application_credentials.create(name='testappcred')
>>> app_cred

Tags:

Revision history for this message

Anastasia (anastasia-macmood) wrote on 2019-06-27:

Yes, we should support it.

I am going through the gap analysis - what authenticate mechanisms Juju supports for each cloud and what mechanisms the clouds themselves support.

I'll add it to our wishist for prioritization.

Changed in juju:
status:	New → Triaged
importance:	Undecided → Wishlist
tags:	added: credentials

Revision history for this message

Dmitrii Shcherbakov (dmitriis) wrote on 2019-06-27:

Thanks for triaging!

Revision history for this message

Tim McNamara (tim-clicks) wrote on 2020-02-29:

This was raised by external users today at the Charm Summit. We should look to move this forward when possible.

tags:

added: frankfurt grnet

Revision history for this message

Pedro Guimarães (pguimaraes) wrote on 2020-07-03:

Had same issue with a customer: they use SAML integration for users and application credential to run API calls through any automation and juju cannot offer it.

Revision history for this message

Pedro Guimarães (pguimaraes) wrote on 2020-07-03:

Here is a trace from bootstrapping with application credentials:
https://pastebin.canonical.com/p/XyzJrBkW9g/

The most interesting part of that log is:
10:31:50 DEBUG cmd supercommand.go:519 error stack:
authentication failed
caused by: requesting token failed
caused by: Resource at https://<REDACTED>/v3/tokens not found
caused by: request (https://<REDACTED>/v3/tokens) returned unexpected status: 404; error info: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>

/build/juju/parts/juju/go/src/github.com/juju/juju/provider/openstack/provider.go:916: authentication failed.
/build/juju/parts/juju/go/src/github.com/juju/juju/environs/bootstrap/prepare.go:133:
/build/juju/parts/juju/go/src/github.com/juju/juju/cmd/juju/commands/bootstrap.go:735:

For some reason, Juju is searching on: https://<REDACTED>/v3/tokens instead of: https://<REDACTED>/v3/auth/tokens
Which is the rule for bionic-stein clouds.

Manually adding "auth/" does not resolve the issue.

Revision history for this message

Graeme Moss (graememoss) wrote on 2021-12-12:

As a end user we use SAML for horizon GUI and Application credentials for CLI and api access.
This option is a must as it allows for better security in not having a a users details inside a juju config. it also allows for if a breach does happen it can be disabled with out breaking other service that use the api.

I would really suggest that this be raised as a higher priority as it offers so much to help the end user with security and easier deployment.

Revision history for this message

Aymen Frikha (aym-frikha) wrote on 2022-01-25:

~field-medium subscribed

Revision history for this message

Heather Lanigan (hmlanigan) wrote on 2022-01-28:

@pguimaraes,

Which version of juju is this?

What do your juju credentials for this openstack look like? What commands did you use to setup the juju credentials for this openstack?

Juju supports openstack identity v3, and will use the auth/tokens if it understands that v3 is in use.

Revision history for this message

John-Paul Robinson (uabjpr) wrote on 2023-03-13:

The ability to authenticate against a cloud with WebSSO seems to be a missing feature in juju. We have a k8s cloud with Keystone for account mgmt with SAML for WebSSO. We can authenticate to the k8s api server using kubectl and the client-go credential plugins.

https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins

This feature uses the Keystone app credentials to retrieve a limited-lifetime token for use with the K8s api server. It relies on an external app that uses the keystone app credentials to retrieve the token and present it to kubectl. This is necessary since our Keystone doesn't store user credentials. It only issues app credentials. Furthermore, k8s doesn't work with the application credentials directly, rather it validates a token issued by keystone (and derived from the app creds).

For juju:

$ juju --version
3.0.3-genericlinux-amd64

Kubectl against our cluster works just fine:

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-world-699cdf74dc-fnjzk 1/1 Running 0 17d
...

The kubectl config directs kubectl to get it's token via an external app. From the user section of the kube config:

users:
- name: username
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1
      args: null
      command: ./app_cred_auth.sh
      env: null
      interactiveMode: Never
      provideClusterInfo: false

The app_cred_auth.sh script uses the Keystone app credentials to retreive a token:

curl -s -i -H "Content-Type: application/json" -d "${data}" "${OS_AUTH_URL}/auth/tokens"

The token is the present to k8s api server to interact with k8s.

Juju needs to do something similar to get a token by way of the app credentials, rather than trying to use the app credentials directly.

My app credentials are loaded in my environment.

$ env | grep OS_
OS_REGION_NAME=RegionOne
OS_INTERFACE=public
OS_AUTH_URL=https://keystone:5000/v3
OS_APPLICATION_CREDENTIAL_SECRET=<secret>
OS_APPLICATION_CREDENTIAL_ID=<id>
OS_AUTH_TYPE=v3applicationcredential
OS_IDENTITY_API_VERSION=3

When juju attempts to detect and load these credentials it complains that it can't understand the user stanza in the kubectl config file:

$ juju autoload-credentials
This operation can be applied to both a copy on this client and to the one on a controller.
No current controller was detected and there are no registered controllers on this client: either bootstrap one or register one.

Looking for cloud and credential information on local client...
ERROR could not detect credentials for provider "kubernetes": failed to read credentials from kubernetes config: configuration for "username" not supported

The ability to authenticate against a cloud with WebSSO seems to be a missing feature in juju.  We have a k8s cloud with Keystone for account mgmt with SAML for WebSSO.  We can authenticate to the k8s api server using kubectl and the client-go credential plugins.

https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins

This feature uses the Keystone app credentials to retrieve a limited-lifetime token for use with the K8s api server.  It relies on an external app that uses the keystone app credentials to retrieve the token and present it to kubectl.  This is necessary since our Keystone doesn't store user credentials. It only issues app credentials. Furthermore, k8s doesn't work with the application credentials directly, rather it validates a token issued by keystone (and derived from the app creds).

For juju:

$ juju --version
3.0.3-genericlinux-amd64

Kubectl against our cluster works just fine:

$ kubectl get pods
NAME                           READY   STATUS    RESTARTS   AGE
hello-world-699cdf74dc-fnjzk   1/1     Running   0          17d
...

The kubectl config directs kubectl to get it's token via an external app.  From the user section of the kube config:

The app_cred_auth.sh script uses the Keystone app credentials to retreive a token:

curl -s -i -H "Content-Type: application/json" -d "${data}" "${OS_AUTH_URL}/auth/tokens"

The token is the present to k8s api server to interact with k8s.

Juju needs to do something similar to get a token by way of the app credentials, rather than trying to use the app credentials directly.

My app credentials are loaded in my environment.

When juju attempts to detect and load these credentials it complains that it can't understand the user stanza in the kubectl config file:

$ juju autoload-credentials 
This operation can be applied to both a copy on this client and to the one on a controller.
No current controller was detected and there are no registered controllers on this client: either bootstrap one or register one.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.