Fernet tokens and Federated Identities result in token scope failures

Bug #1471289 reported by Jesse Pretorius on 2015-07-03
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
High
Marek Denis

Bug Description

When keystone is configured to use fernet tokens and also configured to be a SP for an external IDP then the token data received by nova and other services appear to not contain the right information, resulting in errors from nova-api-os-compute such as:

Returning 400 to user: Malformed request URL: URL's project_id '69f5cff441e04554b285d7772630dec1' doesn't match Context's project_id 'None'

When keystone is switched to use uuid tokens, then everything works as expected.

Further debugging of the request to the nova api shows:

'HTTP_X_USER_DOMAIN_NAME': None,
'HTTP_X_DOMAIN_ID': None,
'HTTP_X_PROJECT_DOMAIN_ID': None,
'HTTP_X_ROLES': '',
'HTTP_X_TENANT_ID': None,
'HTTP_X_PROJECT_DOMAIN_NAME': None,
'HTTP_X_TENANT': None,
'HTTP_X_USER': u'S-1-5-21-2917001131-1385516553-613696311-1108',
'HTTP_X_USER_DOMAIN_ID': None,
'HTTP_X_AUTH_PROJECT_ID': '69f5cff441e04554b285d7772630dec1',
'HTTP_X_DOMAIN_NAME': None,
'HTTP_X_PROJECT_NAME': None,
'HTTP_X_PROJECT_ID': None,
'HTTP_X_USER_NAME': u'S-1-5-21-2917001131-1385516553-613696311-1108'

Comparing the interaction of nova-api-os-compute with keystone for the token validation between an internal user and a federated user, the following is seen:

### federated user ###
2015-07-03 14:43:05.229 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
2015-07-03 14:43:05.265 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 402 x-subject-token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:43:05 GMT content-type: application/json x-openstack-request-id: req-df3dce71-3174-4753-b883-11eb31a67d7c
RESP BODY: {"token": {"methods": ["token"], "expires_at": "2015-07-04T02:43:04.000000Z", "extras": {}, "user": {"OS-FEDERATION": {"identity_provider": {"id": "adfs-idp"}, "protocol": {"id": "saml2"}, "groups": []}, "id": "S-1-5-21-2917001131-1385516553-613696311-1108", "name": "S-1-5-21-2917001131-1385516553-613696311-1108"}, "audit_ids": ["_a6BbQ6mSoGAY2u9NN0tFA"], "issued_at": "2015-07-03T14:43:04.000000Z"}}

### internal user ###
2015-07-03 14:28:31.875 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
2015-07-03 14:28:31.949 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 6691 x-subject-token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:28:31 GMT content-type: application/json x-openstack-request-id: req-6e0ed9f4-46c3-4c79-b444-f72963fc9503
RESP BODY: {"token": {"methods": ["password"], "roles": [{"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "expires_at": "2015-07-04T02:28:31.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "0f491c8551c04cdc804a479af0bf13ec", "name": "demo"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "76c8c3017c954d88a6ad69ee4cb656d6", "name": "test"}, "audit_ids": ["aAN_V0c6SLSI0Rm1hoScCg"], "issued_at": "2015-07-03T14:28:31.000000Z"}}

The data structures that come back from keystone are clearly quite different.

### configuration environment ###

Ubuntu 14.04 OS
nova==12.0.0.0a1.dev51 # commit a4f4be370be06cfc9aa3ed30d2445277e832376f from master branch
keystone==8.0.0.0a1.dev12 # commit a7ca13b687dd284f0980d768b11a3d1b52b4106e from master branch
python-keystoneclient==1.6.1.dev19 # commit d238cc9af4927d1092de207db978536d712af129 from master branch
python-openstackclient==1.5.1.dev11# commit 2d6bc8f4c38dbf997e3e71119f13f0328b4a8669 from master branch
python-novaclient==2.26.1.dev25 # commit 3c2ff0faad8c84777ffe7d9946a1bc4486116084 from master branch
keystonemiddleware==2.0.0
oslo.concurrency==2.1.0
oslo.config==1.12.1
oslo.context==0.4.0
oslo.db==1.12.0
oslo.i18n==2.0.0
oslo.log==1.5.0
oslo.messaging==1.15.0
oslo.middleware==2.3.0
oslo.policy==0.6.0
oslo.serialization==1.6.0
oslo.utils==1.6.0

Keystone is configured as a Shibboleth SP with a trust relationship between it and an ADFS IdP.

The mapping rules are setup as follows - note that the user's default_project_id was added in an attempt to see whether it helped. It does seem to reflect in the HTTP_X_AUTH_PROJECT_ID at the nova api as shown above.

[
  {
    "local": [
      {
        "user": {
          "id": "{0}",
          "name": "{1}_{2}",
          "email": "{3}",
          "default_project_id": "69f5cff441e04554b285d7772630dec1"
        }
      }
    ],
    "remote": [
      {
        "type": "primarysid"
      },
      {
        "type": "givenname"
      },
      {
        "type": "surname"
      },
      {
        "type": "upn"
      }
    ]
  },
  {
    "local": [
      {
        "group": {
          "name": "fedgroup",
          "domain": { "name": "Default" }
        }
      }
    ],
    "remote": [
      {
        "type": "Group",
        "any_one_of": [
          "Domain Users"
        ]
      }
    ]
  }
]

A Shibboleth session shows the following attributes available from the IdP:

Attributes
Group: Domain Users
emailaddress: <email address hidden>
givenname: Super
primarysid: S-1-5-21-2917001131-1385516553-613696311-1108
surname: Ego
upn: <email address hidden>

Role, project, group lists are available here: http://pastebin.com/gcnr9bbT

Changed in keystone:
assignee: nobody → Marek Denis (marek-denis)
Changed in keystone:
status: New → Confirmed
importance: Undecided → High
tags: added: federation fernet
Marek Denis (marek-denis) wrote :

Looks like the problem is with Fernet token formatters that incorrectly handle federated scoped tokens. Those tokens are like ordinary tokens, but they have extra OS-FEDERATION entity inside (however, without groups!).

Fix proposed to branch: master
Review: https://review.openstack.org/202176

Changed in keystone:
status: Confirmed → In Progress
Changed in keystone:
assignee: Marek Denis (marek-denis) → Dolph Mathews (dolph)
Changed in keystone:
assignee: Dolph Mathews (dolph) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Marek Denis (marek-denis)
Changed in keystone:
assignee: Marek Denis (marek-denis) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Marek Denis (marek-denis)
Changed in keystone:
assignee: Marek Denis (marek-denis) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Marek Denis (marek-denis)
Changed in keystone:
assignee: Marek Denis (marek-denis) → Brant Knudson (blk-u)
Brant Knudson (blk-u) on 2015-08-18
Changed in keystone:
assignee: Brant Knudson (blk-u) → Marek Denis (marek-denis)

Reviewed: https://review.openstack.org/202176
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=c5b194300a4b6befbdc75e3142207e5709d42736
Submitter: Jenkins
Branch: master

commit c5b194300a4b6befbdc75e3142207e5709d42736
Author: Marek Denis <email address hidden>
Date: Wed Jul 15 18:18:36 2015 +0200

    Fernet payloads for federated scoped tokens.

    Federated project and domain scoped tokens need to be able to keep
    ``OS-FEDERATION`` info within their payloads.

    Change-Id: I00bbff8066e3842e5a1db4f63ceb7dd9aec083a3
    Closes-Bug: #1471289

Changed in keystone:
status: In Progress → Fix Committed
Changed in keystone:
milestone: none → liberty-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2015-10-15
Changed in keystone:
milestone: liberty-3 → 8.0.0

Change abandoned by Dolph Mathews (<email address hidden>) on branch: stable/kilo
Review: https://review.openstack.org/223863
Reason: Without tests, I'm going to abandon and we'll just have to acknowledge that Fernet will not work with federation in stable/kilo.

no longer affects: keystone/kilo
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers