revoke_api is loaded by default and is causing 401s on horizon switch tenants

Bug #1291099 reported by Meena Pitchiah
18
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Dashboard (Horizon)
Invalid
Undecided
Unassigned
OpenStack Identity (keystone)
Fix Released
Critical
Adam Young

Bug Description

On the top bar, there are two items listed (admin and demo) under tenant switcher.
Log in as admin and try to select demo in the tenant switcher.
Switching to demo results in "unauthorized Access " HTTP error 401.

Revision history for this message
Meena Pitchiah (meena-pitchiah) wrote :
Revision history for this message
Walt Boring (walter-boring) wrote :

As a side note, after you try and switch projects and get the HTTP error 401, horizon is completely unusable. You can't go to any horizon page and get it to work after that. The only remedy is to kill all of your browser processes/nuke all cookies and start over.

Changed in horizon:
status: New → Confirmed
Revision history for this message
Michael Denny (michael-denny) wrote :

This is very cumbersome working with multiple tenants should be be fixed quickly

Revision history for this message
David Lapsley (dlapsley) wrote :

Folks, it turns out this is an issue with the keystone backend version. With a default devstack install, there are no keystone settings, so horizon defaults to version "2.0". In this case, we see the behavior above with Keystone complaining that it can't validate the horizon supplied token for the switch_tenant() auth view.

If you set your backend to version 3 using the settings below in your local_settings.py, the tenant_switcher works as it should:

OPENSTACK_API_VERSIONS = {
    "identity": 3
}
OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "member"

Cheers, Dave.

Revision history for this message
David Lapsley (dlapsley) wrote :

I'm digging more into the code to see why v2 is not working.. Sounds like it must be either a config issue or in how we are calling the keystone API.

Revision history for this message
David Lapsley (dlapsley) wrote :

There appear to be a number of issues here:

1. django_openstack_auth recently set its default openstack identity api version to "3" from "2.0". By default horizon defaults to "2.0" if nothing is specified in local_settings.py. This needs to be fixed. There are a number of ways to address this. I think the easiest and cleanest way would be to add the settings above to settings.py as default. I would be happy to upload a patch set to do this.

2. Keystone appears to have an issue with v2.0 project switching requests. If you login as "demo" user in a devstack environment with the explicit 2.0 settings (as shown below), you will be able to switch projects from "demo" to "invisible_to_admin". However, if you login as "admin" user and attempt to switch to "demo" project, you will get a 401. The Keystone log will show that Keystone failed to validate the token passed to it (although tracing and db lookup shows the correct token makes it to keystone). Furthermore, django_openstack_auth successfully completes its part in switch_tenant(). The reason this is happening is because Keystone is not able to validate tokens associated with an "admin" role during the switch projects use case in v2.0 mode. If you change the "admin" user's role to Member only for the "demo" project, it is possible to switch projects from the "admin" project to the "demo" project. (You have to manually set the redirect URL to "next=project" in your browser, as there is another bug there).

At this point, it looks a lot like a keystone v2.0 compat issue. I have verified that it was not caused but the Accordion Nav patch.

Here are the settings for explicit v2.0 support in horizon (required now otherwise there will be a mismatch between django_openstack_auth and horizon):

OPENSTACK_API_VERSIONS = {
    "identity": 2.0
}
OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v2.0" % OPENSTACK_HOST
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "Member"

So, it seems like there are at least a few important questions here and at least two issues to manage:

1. How important is v2.0 compat
2. How best to ensure that django_openstack_auth and horizon have the same default identity api version?

Hopefully this all makes sense. Would appreciate any thoughts on this and where we go from here.

Cheers, Dave.

Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

This bug appears to be related to the new revocation_api validation.

Changed in keystone:
assignee: nobody → Morgan Fainberg (mdrnstm)
importance: Undecided → Critical
Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

Continuation of comment #7:

This is within the Keystone server not within horizon. I'm in the process of looking into why this is occurring in the way that it is.

Changed in keystone:
milestone: none → icehouse-rc1
status: New → Triaged
Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

This is because the revoke_api has changed how we handle revoked tokens and it was being loaded by default incorrectly. The solution for this bug will be to eliminate the revoke_api loading by default.

A secondary bug will be opened to work towards correcting the behavior where revoking a token (delete) from a token chain (e.g. Unscoped Token -> Scoped Token, the scoped token being revoked would also revoke the unscoped token) causes the entire chain of tokens to be revoked. In some cases this is desirable, in others it is not.

Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

I am removing the tag of this from horizon as it is not a Horizon bug.

Changed in horizon:
status: Confirmed → Invalid
summary: - tenant switcher on the horizon dashboard results in HTTP 401
+ revoke_api is loaded by default and is causing 401s on horizon switch
+ tenants
Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

Associated bug that addresses the logic issues that highlighted that revoke_api was loaded by default: https://bugs.launchpad.net/keystone/+bug/1292283

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/80441

Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

I will associate horizon to the other bug so this one can be closed as it is an RC blocker for keystone.

Changed in keystone:
status: Triaged → In Progress
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Adam Young (ayoung)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/80441
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=7c9746c49b7518f2a000f7e4822e822c6eeee81d
Submitter: Jenkins
Branch: master

commit 7c9746c49b7518f2a000f7e4822e822c6eeee81d
Author: Adam Young <email address hidden>
Date: Thu Mar 13 21:11:54 2014 -0400

    Don't automatically enable revocation events.

    Cuts any direct imports of revoke.model as that
    triggers the dependency registration.

    In order to fix a dependecy resolution issue, changes the syncronziation to
    using the same sort of chaching mechanism as the other drivers.

    Adds the ability to Lazy activate providers for future or optional dependency
    resolution.

    Closes-Bug: 1291099
    Related-Bug: 1292283
    Change-Id: I0db36b295c2040ec1fb248cf75dc55c44c059211

Changed in keystone:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.