Attribute error on Token object when using domain scoped token

[Brad Pokorny]

When making a policy check from Django OpenStack Auth with a domain scoped token, I'm seeing this error:

2016-02-19 19:54:20.935905 TypeError: 'Token' object has no attribute '__getitem__'

This only occurs when using the latest v3 policy file from Keystone [1], which currently contains this line:

    "cloud_admin": "role:admin and (token.is_admin_project:True or domain_id:admin_domain_id)",

When I revert that line back to what it is for stable/liberty, the issue goes away:

    "cloud_admin": "rule:admin_required and domain_id:admin_domain_id",

So there may be a case that's currently not handled when using the "token" string in policy files.

Info on variables that are set when calling the enforce method of oslo.policy [2]:

"if not enforcer_scope.enforce(action, target, credentials):"

(Pdb) action
'identity:list_projects'
(Pdb) target
{'user_id': u'b2db130f48ac49c28d055ad65248f07e', 'user.domain_id': u'aca9b87dde25410da94726bb0c6a728d', 'group.domain_id': u'aca9b87dde25410da94726bb0c6a728d', 'project.domain_id': u'aca9b87dde25410da94726bb0c6a728d', 'project_id': u'e8758f97a4ca4ec585e8d940c1acc125', 'domain_id': u'aca9b87dde25410da94726bb0c6a728d'}
(Pdb) credentials
{'username': u'domain_admin', 'token': <openstack_auth.user.Token object at 0xb382a78c>, 'project_name': None, 'user_id': u'b2db130f48ac49c28d055ad65248f07e', 'roles': [u'admin'], 'is_admin': True, 'project_id': None, 'domain_id': u'aca9b87dde25410da94726bb0c6a728d'}

Version of oslo.policy:

$ pip show oslo.policy
---
Metadata-Version: 2.0
Name: oslo.policy
Version: 1.4.0
Summary: Oslo Policy library

Version of DOA:

$ pip show django-openstack-auth
---
Metadata-Version: 2.0
Name: django-openstack-auth
Version: 2.1.1
Summary: Django authentication backend for use with OpenStack Identity

[1] https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json
[2] https://github.com/openstack/oslo.policy/blob/master/oslo_policy/policy.py#L515

Full stack trace from Horizon when using the domain scoped token and logging in as a domain admin of a domain other that the default domain (requires this patch to reproduce from Horizon: https://review.openstack.org/#/c/148082/):

2016-02-19 19:54:20.935395 File "/opt/stack/horizon/openstack_dashboard/wsgi/../../openstack_dashboard/dashboards/identity/projects/views.py", line 84, in get_data
2016-02-19 19:54:20.935428 self.request):
2016-02-19 19:54:20.935449 File "/opt/stack/horizon/openstack_dashboard/wsgi/../../openstack_dashboard/policy.py", line 24, in check
2016-02-19 19:54:20.935470 return policy_check(actions, request, target)
2016-02-19 19:54:20.935489 File "/usr/local/lib/python2.7/dist-packages/openstack_auth/policy.py", line 155, in check
2016-02-19 19:54:20.935510 enforcer[scope], action, target, domain_credentials)
2016-02-19 19:54:20.935530 File "/usr/local/lib/python2.7/dist-packages/openstack_auth/policy.py", line 169, in _check_credentials
2016-02-19 19:54:20.935559 if not enforcer_scope.enforce(action, target, credentials):
2016-02-19 19:54:20.935579 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py", line 551, in enforce
2016-02-19 19:54:20.935599 result = self.rules[rule](target, creds, self)
2016-02-19 19:54:20.935619 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 160, in __call__
2016-02-19 19:54:20.935639 if rule(target, cred, enforcer):
2016-02-19 19:54:20.935658 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 204, in __call__
2016-02-19 19:54:20.935679 return enforcer.rules[self.match](target, creds, enforcer)
2016-02-19 19:54:20.935698 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 125, in __call__
2016-02-19 19:54:20.935727 if not rule(target, cred, enforcer):
2016-02-19 19:54:20.935747 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 160, in __call__
2016-02-19 19:54:20.935767 if rule(target, cred, enforcer):
2016-02-19 19:54:20.935786 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 311, in __call__
2016-02-19 19:54:20.935806 return self._find_in_dict(creds, path_segments, match)
2016-02-19 19:54:20.935826 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 292, in _find_in_dict
2016-02-19 19:54:20.935846 return self._find_in_dict(test_value, path_segments, match)
2016-02-19 19:54:20.935866 File "/usr/local/lib/python2.7/dist-packages/oslo_policy/_checks.py", line 283, in _find_in_dict
2016-02-19 19:54:20.935886 test_value = test_value[key]
2016-02-19 19:54:20.935905 TypeError: 'Token' object has no attribute '__getitem__'

[Dan Nguyen (daniel-a-nguyen)]

I think leaving 'token.is_admin_project:True' out of horizon and django-openstack-auth is probably the best work around for now.

Thanks for submitting this Brad.

[Adam Young (ayoung)]

Would have been fixed long ago if this patch had been merged:

https://review.openstack.org/#/c/165908/

[Brad Pokorny (bpokorny)]

I was able to reproduce this in devstack with the keystone v3 sample policy file, without going through DOA, and with this code:

http://paste.openstack.org/show/488090/

I think what Adam is saying is this line in the policy:

    "cloud_admin": "role:admin and (token.is_admin_project:True or domain_id:admin_domain_id)",

is known to be incompatible with some v3 operations, and we would need https://review.openstack.org/#/c/165908/ to silently disregard the incompatibility and move on.

Please correct me if I'm wrong there.

[Steve Martinelli (stevemar)]

Related bug 1571875 and bug 1459884

[Steve Martinelli (stevemar)]

Another duplicate bug: https://bugs.launchpad.net/keystone/+bug/1588190

[Steve Martinelli (stevemar)]

So https://review.openstack.org/#/c/165908/ exists, but it will likely cover up proper failures too, any other ideas? Should we be trying to fix the issue, or just document that this is going to be a limitation?

Brant: Isn't it safer to write your rules correctly to begin with rather than ignoring errors?

[Sean Carlisle (sean-carlisle)]

We're able to get around this like previous commenters by removing token.is_admin_project:True for Horizon's Keystone policy file. However, keeping it in Keystone's copy of the policy file necessary as we utilize applications that are not yet domain aware.

[Lance Bragstad (lbragstad)]

Another bug was reported today (bug 1645632) that looks very similar to this. I marked bug 1645632 as a duplicate and updating here so that it gets visibility.

[Jay Jahns (jjahns)]

Thanks Lance.

Reviewing the data in this bug, I don't think suppressing the failure is the right course of action, so the is_admin_project adjustment will be required.

We really should look at fixing this at some point and not working around the issue like we've been doing. Things like this can be a real nightmare to some operators if they don't know to look here or the community resources.

[Steve Martinelli (stevemar)]

This is an improperly written keystone rule. Marking oslo.policy as Invalid.

The fix is to change:

  token.is_admin_project:True

to:

  is_admin_project:True

Note to affected users, we typically do not backport changes to config files, so please update the policy files accordingly.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/411562

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/411563

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/411562
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3845e36f0dc546bd7b3ca1a69c590b3031616acf
Submitter: Jenkins
Branch: master

commit 3845e36f0dc546bd7b3ca1a69c590b3031616acf
Author: Steve Martinelli <email address hidden>
Date: Thu Dec 15 14:23:13 2016 -0800

    expose v3policy failure with is_admin_token

    The cloud_admin rule is incorrectly written, and will result in
    a TypeError when any action is taken that uses the cloud_admin
    rule (most of our APIs). This change exposes that error by
    attempting to load and enforce the policy, this is done for
    both sample policies we provide.

    Partial-Bug: 1547684

    Change-Id: If039bfb13741ee5a4b6b9c34b45ceafcfb5b342e

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/411563
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ef48072d94f780ebaacee8c3ddf02a68193fa74d
Submitter: Jenkins
Branch: master

commit ef48072d94f780ebaacee8c3ddf02a68193fa74d
Author: Steve Martinelli <email address hidden>
Date: Thu Dec 15 17:48:16 2016 -0800

    Fix cloud_admin rule and ensure only project tokens can be cloud admin

    The current rule fails to load with oslo.policy, the correct
    value used to determine the admin project for the cloud_admin should
    simply be: `is_admin_project:True`, since that is what is stored
    in oslo.context.

    This problem was masking a more serious issue that domain admin tokens
    could be misinterpreted as cloud admin tokens.

    Change-Id: I3ea562c01e06e6c519fdaec3ab6e1dac204ced71
    Closes-Bug: 1547684
    Closes-Bug: 1651989

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 11.0.0.0b3 development milestone.

[David Lyle (david-lyle)]

Just noticed this was otherwise fixed. My copy of the policy file was stale.

[Jamie Lennox (jamielennox)]

I think the problem here is that keystone dumps the entire token into the policy file, hence expecting people to understand the fields in the token. This has meant other projects have copied the pattern.

Keystone should not be doing this and other projects should absolutely not do it.

[OpenStack Infra (hudson-openstack)]

Change abandoned by ayoung (<email address hidden>) on branch: master
Review: https://review.openstack.org/165908
Reason: Most important parts of this have been integrated in other patches.