revoke_api is loaded by default and is causing 401s on horizon switch tenants

[Meena Pitchiah]

On the top bar, there are two items listed (admin and demo) under tenant switcher.
Log in as admin and try to select demo in the tenant switcher.
Switching to demo results in "unauthorized Access " HTTP error 401.

[Walt Boring (walter-boring)]

As a side note, after you try and switch projects and get the HTTP error 401, horizon is completely unusable. You can't go to any horizon page and get it to work after that. The only remedy is to kill all of your browser processes/nuke all cookies and start over.

[Michael Denny (michael-denny)]

This is very cumbersome working with multiple tenants should be be fixed quickly

[David Lapsley (dlapsley)]

Folks, it turns out this is an issue with the keystone backend version. With a default devstack install, there are no keystone settings, so horizon defaults to version "2.0". In this case, we see the behavior above with Keystone complaining that it can't validate the horizon supplied token for the switch_tenant() auth view.

If you set your backend to version 3 using the settings below in your local_settings.py, the tenant_switcher works as it should:

OPENSTACK_API_VERSIONS = {
    "identity": 3
}
OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "member"

Cheers, Dave.

[David Lapsley (dlapsley)]

I'm digging more into the code to see why v2 is not working.. Sounds like it must be either a config issue or in how we are calling the keystone API.

[David Lapsley (dlapsley)]

There appear to be a number of issues here:

1. django_openstack_auth recently set its default openstack identity api version to "3" from "2.0". By default horizon defaults to "2.0" if nothing is specified in local_settings.py. This needs to be fixed. There are a number of ways to address this. I think the easiest and cleanest way would be to add the settings above to settings.py as default. I would be happy to upload a patch set to do this.

2. Keystone appears to have an issue with v2.0 project switching requests. If you login as "demo" user in a devstack environment with the explicit 2.0 settings (as shown below), you will be able to switch projects from "demo" to "invisible_to_admin". However, if you login as "admin" user and attempt to switch to "demo" project, you will get a 401. The Keystone log will show that Keystone failed to validate the token passed to it (although tracing and db lookup shows the correct token makes it to keystone). Furthermore, django_openstack_auth successfully completes its part in switch_tenant(). The reason this is happening is because Keystone is not able to validate tokens associated with an "admin" role during the switch projects use case in v2.0 mode. If you change the "admin" user's role to Member only for the "demo" project, it is possible to switch projects from the "admin" project to the "demo" project. (You have to manually set the redirect URL to "next=project" in your browser, as there is another bug there).

At this point, it looks a lot like a keystone v2.0 compat issue. I have verified that it was not caused but the Accordion Nav patch.

Here are the settings for explicit v2.0 support in horizon (required now otherwise there will be a mismatch between django_openstack_auth and horizon):

OPENSTACK_API_VERSIONS = {
    "identity": 2.0
}
OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v2.0" % OPENSTACK_HOST
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "Member"

So, it seems like there are at least a few important questions here and at least two issues to manage:

1. How important is v2.0 compat
2. How best to ensure that django_openstack_auth and horizon have the same default identity api version?

Hopefully this all makes sense. Would appreciate any thoughts on this and where we go from here.

Cheers, Dave.

[Morgan Fainberg (mdrnstm)]

This bug appears to be related to the new revocation_api validation.

[Morgan Fainberg (mdrnstm)]

Continuation of comment #7:

This is within the Keystone server not within horizon. I'm in the process of looking into why this is occurring in the way that it is.

[Morgan Fainberg (mdrnstm)]

This is because the revoke_api has changed how we handle revoked tokens and it was being loaded by default incorrectly. The solution for this bug will be to eliminate the revoke_api loading by default.

A secondary bug will be opened to work towards correcting the behavior where revoking a token (delete) from a token chain (e.g. Unscoped Token -> Scoped Token, the scoped token being revoked would also revoke the unscoped token) causes the entire chain of tokens to be revoked. In some cases this is desirable, in others it is not.

[Morgan Fainberg (mdrnstm)]

I am removing the tag of this from horizon as it is not a Horizon bug.

[Morgan Fainberg (mdrnstm)]

Associated bug that addresses the logic issues that highlighted that revoke_api was loaded by default: https://bugs.launchpad.net/keystone/+bug/1292283

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/80441

[Morgan Fainberg (mdrnstm)]

I will associate horizon to the other bug so this one can be closed as it is an RC blocker for keystone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/80441
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=7c9746c49b7518f2a000f7e4822e822c6eeee81d
Submitter: Jenkins
Branch: master

commit 7c9746c49b7518f2a000f7e4822e822c6eeee81d
Author: Adam Young <email address hidden>
Date: Thu Mar 13 21:11:54 2014 -0400

    Don't automatically enable revocation events.

    Cuts any direct imports of revoke.model as that
    triggers the dependency registration.

    In order to fix a dependecy resolution issue, changes the syncronziation to
    using the same sort of chaching mechanism as the other drivers.

    Adds the ability to Lazy activate providers for future or optional dependency
    resolution.

    Closes-Bug: 1291099
    Related-Bug: 1292283
    Change-Id: I0db36b295c2040ec1fb248cf75dc55c44c059211