x509 authentication doesn't work with auto-provisioning

[Lance Bragstad]

Keystone supports the ability to authenticate users who have an x509 certificate [0]. Attributes from the certificate are parsed using an SSL library and are passed to keystone (mod_ssl in the apache case). Keystone uses the certificate attributes like attributes from a SAML assertion and runs it through a mapping.

Keystone also supports auto-provisioning through mappings [1]. This is a mapping with a special syntax that let's keystone know it should create the underlying projects and role assignments for that user, in addition to the shadow user.

It appears that none of the auto-provisioning works when authenticating with x509 certificates. The following are the steps I took to determine this.

1. ) Deploy devstack with the tls-proxy service enabled

disable_all_services
enable_service key
enable_service tls-proxy
enable_service horizon
enable_service mysql
enable_service tempest
enable_service rabbit

2.) Remove http-services-tls-proxy.conf Apache configuration

$ sudo rm /etc/apache2/sites-available/http-services-tls-proxy.conf

For some reason, this doesn't work with keystone and SSL.

3.) Configure SSL for keystone in Apache configuration

<VirtualHost *:443>
    SSLEngine On
    SSLCertificateFile /opt/stack/data/devstack-cert.pem
    SSLCACertificateFile /opt/stack/data/CA/root-ca/cacert.pem
    SSLOptions +StdEnvVars
    SSLVerifyClient optional
    SSLUserName SSL_CLIENT_S_DN_CN
    SetEnv REMOTE_DOMAIN openstack
</Virtualhost>

4.) Configure x509 authentication in keystone.conf

[tokenless_auth]
# trusted issuer comes from the Issuer of the SSLCACertificateFile
# from the Apache configuration, where $CERT_PATH is SSLCACertificateFile.
# Use openssl to figure out the Issuer and reformat the ordering of the string
# to be as follows.
trusted_issuer = CN=Root CA,OU=DevStack Certificate Authority,O=OpenStack

[auth]
methods = password,token,external
external = Domain

5.) Create a certificate request, private key, and certificate for a user

$ openssl req -out john.csr -new -newkey rsa:2048 -nodes -keyout john.key

When prompted for the organization (O) enter the domain for the user.
A domain with the same name as the organizational unit will need to be
created in keystone. The common name of the certificate request will be
the user's username within keystone, depending on the specific mapping.

$ openssl x509 -req -in john.csr -CA $ROOT_CA -CAkey $ROOT_KEY -days 365 -out john.pem -CAcreateserial

6.) Create the identity provider and mapping in keystone

The identity provider ID is the hashed trusted_issuer string from the configuration file:

./hash 'CN=Root CA,OU=DevStack Certificate Authority,O=OpenStack'
ad9b5af1ba36ffc36e1fbf7af5e83e25aebf66bbfefc12eed0313a875e6f9663

https://gist.github.com/lbragstad/5338e8bfdcc1158ceaedffd4036e671e

$ openstack identity provider create ad9b5af1ba36ffc36e1fbf7af5e83e25aebf66bbfefc12eed0313a875e6f9663
$ openstack mapping create x509map --rules rules.json
$ openstack federation protocol create x509 --mapping x509map --identity-provider ad9b5af1ba36ffc36e1fbf7af5e83e25aebf66bbfefc12eed0313a875e6f9663

The following is the rules.json I used:

https://pasted.tech/pastes/a825b80d51c6e01d87d0b692a1c1d30c275b72b4.raw

7.) Authenticate for a token

When I authenticate for a token using the certificate I created, I get an unscoped token. Also, none of the auto-provisioned resources are created through the mapping. The expected behavior is that the auto-provisioning feature would handle the creation of those resources and I'd be able to get a scoped token.

https://pasted.tech/pastes/e51f35565c923c2469dc44ec5f42cca8ecc1cf9f.raw

[0] https://docs.openstack.org/keystone/latest/admin/configure_tokenless_x509.html#create-a-map

[Lance Bragstad (lbragstad)]

This might not be specific to this bug, but I did notice that we are calling the external authentication plugin twice, which we probably don't need to do.

I was reading the code and noticed we special case the external authentication plugin, but then we also rely on a factory to return the authentication plugin needed based on the methods supplied in the request.

Here are the logging changes I made:

https://pasted.tech/pastes/9df194511c3fbd509f216eb88b70ef4ee0479dad.raw

And this is the logs I captured when authenticating with an x509 certificate:

https://pasted.tech/pastes/a2197a098d9ad1d5397561ef0e63f3876078ffb2.raw

I'm not sure yet if this causes issues in how the requests are handled, but it doesn't look like it. More-or-less a performance issue and extra, un-needed calls.

[Lance Bragstad (lbragstad)]

After digging into this, I think I figured out why this is happening.

In the example, I configured keystone to have an external authentication method. This allows users to use x509 certificates to authenticate, offloading the validation of the certificate to apache plugins. The data is then passed into keystone in the form of a request context.

Keystone process the request context using middleware, which executes before the actual code in keystone's authentication API. The middleware determines if the request was authenticated using tokenless authentication methods and fires off a call to the tokenless authentication helper class, which handles its own mapping logic[0][1][2][3][4][5]. The tokenless helper class doesn't reuse the Mapped plugin that gets used for other protocols, like SAML. The Mapped plugin is what actually has support for auto-provisioning [6].

The auto-provisioning feature was implemented after the x509 authentication plugin. So this was likely due to the fact that there were two mapping implementations, and one wasn't as well known.

We could try and get all protocols using the same mapping implementation moving forward. This would make it so we're less susceptible to these kinds of issues moving forward and makes using x509 more useful.

[0] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/server/flask/request_processing/middleware/auth_context.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n330
[1] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/server/flask/request_processing/middleware/auth_context.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n360
[2] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/server/flask/request_processing/middleware/auth_context.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n436
[3] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/server/flask/request_processing/middleware/auth_context.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n248
[4] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/server/flask/request_processing/middleware/auth_context.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n258
[5] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/tokenless_auth.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n96
[6] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/auth/plugins/mapped.py?id=e647d6f69762523d0dfa28137a9f11010b550e72#n109