ec2 credential "trust_id" can be updated to null

[kay]

Similar to https://bugs.launchpad.net/keystone/+bug/1872733 and https://bugs.launchpad.net/keystone/+bug/1872753. If ec2 credentials were created within a trust_id scope, it is still possible to set these credentials' "trust_id" to "null" using:

curl -X PATCH https://keystone/v3/credentials/3c2b3265350c6da3a18a143fbe975ca4a8ed88a6f8c6dacc2494a5c1287ba66f -H 'Accept: application/json' -H 'Content-Type: application/json' -H "X-Auth-Token: ***" -d'{
  "credential": {
    "blob": "{\"access\": \"ffe6fc21b47c4d87befc95ad070c3b7a\", \"secret\": \"530196cd097e4a7ca9df7258aa89ff0e\", \"trust_id\": null}"
  }
}'

Note "null" in blob.

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete
security advisory task has been added while the core security
reviewers for the affected project or projects confirm the bug and
discuss the scope of any vulnerability along with potential
solutions.

[Colleen Murphy (krinkle)]

I have verified that this is true, a credential created using a trust scope has the trust_id attribute in the blob and this can be PATCHed to set trust_id to null. However, I'm unclear on the consequences of this. No matter the content of the trust_id value of the blob, a token request using the EC2 credential does not have trust information in it, see attached example. Only the admin or owner can modify the credential. Can you explain with examples how this could be exploited?

[kay (kay-diam)]

> a token request using the EC2 credential does not have trust information in it, see attached example.

This sounds insecure. What is the purpose of the "trust_id" attribute if it is not considered? Looks like it is another security flaw. I suppose a proper solution would be to add a scope field for all possible SUB auth methods, and consider this field when a new openstack token is generated during authorization:
* trust scope
* oauth1 scope
* application credential scope
* etc.

Thoughts?

[Colleen Murphy (krinkle)]

I did some spelunking and it looks like the trust_id used to be used to add the correct roles to a token requested with an ec2 credential, and that functionality was mistakenly removed in https://review.opendev.org/389365. The consequence is documented in https://bugs.launchpad.net/keystone/+bug/1872735 so I propose we close this as a duplicate of that one.

[Colleen Murphy (krinkle)]

Actually, the fixes will be different - this needs a change to the CRUD of the credential, 1872735 needs a change to the token validation, so not a duplicate. Will mark as medium severity since the main escalation path is 1872735.

[Colleen Murphy (krinkle)]

Patch attached. This depends on the patch for https://bugs.launchpad.net/keystone/+bug/1872735 since it also accounts for app_cred_id.

[kay (kay-diam)]

There is a typo in release notes:

- used by keystone to determine the scoped allowed
+ used by keystone to determine the scope allowed

[Colleen Murphy (krinkle)]

Thanks for the review, new patch includes the typo fix.

[Gage Hugo (gagehugo)]

Revised patch set looks good to me, thanks Colleen.

[Lance Bragstad (lbragstad)]

The patchset in comment #8 looks good.

[Colleen Murphy (krinkle)]

Is a CVE being requested for this? How can we move this forward?

[Gage Hugo (gagehugo)]

Since the exploitative method is described in https://bugs.launchpad.net/keystone/+bug/1872735 and will be fixed, IMO this can be marked as Class D as per the VMT Taxonomy[0].

[0] https://security.openstack.org/vmt-process.html#incident-report-taxonomy

[Jeremy Stanley (fungi)]

Sounds like we should at least delay disclosure of this until bug 1872735 is already public (we could plan to just switch them both at the same time).

[Colleen Murphy (krinkle)]

Agreed on both counts, the exploit is covered by the other bug, but also this can't be safely disclosed separately from the other bug.

[Colleen Murphy (krinkle)]

I've reworked this patch to include fixes for bug 1872733, bug 1872735 and bug 1872755 and to resolve the merge conflict arising from the fix for bug 1872737. These patches are independently appliable but they won't pass unit tests without also applying the fix for 1873290. I've attached them on bug 1872735.

[Gage Hugo (gagehugo)]

The embargo for this bug has expired, will be making public and fixes will be available in gerrit here shortly.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/725886

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/725888

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/725891

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/725893

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/725895

[Jeremy Stanley (fungi)]

I've set our advisory task to Won't Fix on this one, as no advisory is required with the fix for bug 1872735 effectively preventing the path to exploitation.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/726046

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725888
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=2f2736ebb267c757ad77fcf25ee0aaeefab2a09d
Submitter: Zuul
Branch: stable/ussuri

commit 2f2736ebb267c757ad77fcf25ee0aaeefab2a09d
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Change-Id: I39d0d705839fbe31ac518ac9a82959e108cb7c1d
    Closes-bug: #1872733
    Closes-bug: #1872755
    Closes-bug: #1872735
    (cherry picked from commit 37e9907a176dad6843819b1bec4946c3aecc4548)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725886
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=37e9907a176dad6843819b1bec4946c3aecc4548
Submitter: Zuul
Branch: master

commit 37e9907a176dad6843819b1bec4946c3aecc4548
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Change-Id: I39d0d705839fbe31ac518ac9a82959e108cb7c1d
    Closes-bug: #1872733
    Closes-bug: #1872755
    Closes-bug: #1872735

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/726435

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725891
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=54590544fb7a2cd65e23fce8abcf77296352cd29
Submitter: Zuul
Branch: stable/train

commit 54590544fb7a2cd65e23fce8abcf77296352cd29
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9, test helper
    in 52da4d0e12):
          keystone/api/credentials.py
          keystone/tests/unit/test_v3_application_credential.py
          keystone/tests/unit/test_v3_credential.py

    Depends-on: htt...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725893
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=206392a40ce3da3d389e5e7ec721ada2737b84da
Submitter: Zuul
Branch: stable/stein

commit 206392a40ce3da3d389e5e7ec721ada2737b84da
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9):
          keystone/api/credentials.py
          keystone/tests/unit/test_v3_application_credential.py
          keystone/tests/unit/test_v3_credential.py

    Change-Id: I39d0d705839fbe31ac518ac9a82959e108...

[Nick Tait (nickthetait)]

Too late to matter, but I also support Class D here.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725895
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=53d1ccb8a1bdbb5aa0efaacf9739b1a6f436e191
Submitter: Zuul
Branch: stable/rocky

commit 53d1ccb8a1bdbb5aa0efaacf9739b1a6f436e191
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9):
          keystone/api/credentials.py
          keystone/tests/unit/test_v3_application_credential.py
          keystone/tests/unit/test_v3_credential.py

    Conflicts due to flask reorg:
            keys...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/726435
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=487c7276c7608fb11086b9875b0d7cc7cf594a5a
Submitter: Zuul
Branch: stable/queens

commit 487c7276c7608fb11086b9875b0d7cc7cf594a5a
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9):
          keystone/api/credentials.py
          keystone/tests/unit/test_v3_application_credential.py
          keystone/tests/unit/test_v3_credential.py

    Conflicts due to flask reorg:
            key...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/726046
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=a405e4b71d7de31e81a01f07e02f189650eb66fe
Submitter: Zuul
Branch: stable/pike

commit a405e4b71d7de31e81a01f07e02f189650eb66fe
Author: Colleen Murphy <email address hidden>
Date: Tue Apr 14 16:47:44 2020 -0700

    Fix security issues with EC2 credentials

    This change addresses several issues in the creation and use of EC2/S3
    credentials with keystone tokens.

    1. Disable altering credential owner attributes or metadata

    Without this patch, an authenticated user can create an EC2 credential
    for themself for a project they have a role on, then update the
    credential to target a user and project completely unrelated to them. In
    the worst case, this could be the admin user and a project the admin
    user has a role assignment on. A token granted for an altered credential
    like this would allow the user to masquerade as the victim user. This
    patch ensures that when updating a credential, the new form of the
    credential is one the acting user has access to: if the system admin
    user is changing the credential, the new user ID or project ID could be
    anything, but regular users may only change the credential to be one
    that they still own.

    Relatedly, when a user uses an application credential or a trust to
    create an EC2 credential, keystone automatically adds the trust ID or
    application credential ID as metadata in the EC2 access blob so that it
    knows how the token can be scoped when it is used. Without this patch, a
    user who has created a credential in this way can update the access blob
    to remove or alter this metadata and escalate their privileges to be
    fully authorized for the trustor's, application credential creator's, or
    OAuth1 access token authorizor's privileges on the project. This patch
    fixes the issue by simply disallowing updates to keystone-controlled
    metadata in the credential.

    2. Respect token roles when creating EC2 credentials

    Without this patch, a trustee, an application credential user, or an
    OAuth1 access token holder could create an EC2 credential or an
    application credential using any roles the trustor, application
    credential creator, or access token authorizor had on the project,
    regardless of whether the creator had delegated only a limited subset of
    roles. This was because the trust_id attribute of the EC2 access blob
    was ignored, and no metadata for the application credential or access
    token was recorded either. This change ensures that the access
    delegation resource is recorded in the metadata of the EC2 credential
    when created and passed to the token provider when used for
    authentication so that the token provider can look up the correct roles
    for the request.

    Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9):
          keystone/api/credentials.py
          keystone/tests/unit/test_v3_application_credential.py
          keystone/tests/unit/test_v3_credential.py

    Conflicts due to flask reorg:
            keyst...

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone pike-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone rocky-eol release.