[OSSA-2019-006] Credentials API allows listing and retrieving of all users credentials (CVE-2019-19687)

[Daniel 'f0o' Preussker]

Tested against Stein and Train.

# User creating a credential, i.e totp or similar
$ OS_CLOUD=1 openstack token issue
| project_id | c3caf1b55bb84b78a795fd81838e5160
| user_id | 9971b0f13d2d4a578212d028a53c3209
$ OS_CLOUD=1 openstack credential create --type test 9971b0f13d2d4a578212d028a53c3209 test-data
$ OS_CLOUD=1 openstack credential list
+----------------------------------+------+----------------------------------+-----------+------------+
| ID | Type | User ID | Data | Project ID |
+----------------------------------+------+----------------------------------+-----------+------------+
| 0a3a2d3b7dad4886b0bbf61b6cd7d2b0 | test | 9971b0f13d2d4a578212d028a53c3209 | test-data | None |
+----------------------------------+------+----------------------------------+-----------+------------+

# Different User but same Project
$ OS_CLOUD=2 openstack token issue
| project_id | c3caf1b55bb84b78a795fd81838e5160
| user_id | 6b28a0b073fc4ac7843f33190ebc5c3c
$ OS_CLOUD=2 openstack credential list
+----------------------------------+------+----------------------------------+-----------+------------+
| ID | Type | User ID | Data | Project ID |
+----------------------------------+------+----------------------------------+-----------+------------+
| 0a3a2d3b7dad4886b0bbf61b6cd7d2b0 | test | 9971b0f13d2d4a578212d028a53c3209 | test-data | None |
+----------------------------------+------+----------------------------------+-----------+------------+

# Different User and Different Project
$ OS_CLOUD=3 openstack token issue
| project_id | d43f20ae5a7e4f36b701710277384401
| user_id | 2e48f1a7d1474391a826a2b9700e5949
$ OS_CLOUD=3 openstack credential list
+----------------------------------+------+----------------------------------+-----------+------------+
| ID | Type | User ID | Data | Project ID |
+----------------------------------+------+----------------------------------+-----------+------------+
| 0a3a2d3b7dad4886b0bbf61b6cd7d2b0 | test | 9971b0f13d2d4a578212d028a53c3209 | test-data | None |
+----------------------------------+------+----------------------------------+-----------+------------+

As shown anyone who's authenticated can retrieve any credentials including their 'secret'.

This is a rather severe information disclosure vulnerability and completely defies the purpose of TOTP or MFA as these credentials are not kept secure or private whatsoever.

If Auth-rules are configured allow login with only 'topt' it would be extremely easy to assume a different user's identity.

A CVE should be issued for this. I can take care of that paperwork.

Versions affected and tested:

Train/ubuntu:
$ dpkg -l | grep keystone
ii keystone 2:16.0.0-0ubuntu1~cloud0 all OpenStack identity service - Daemons
ii keystone-common 2:16.0.0-0ubuntu1~cloud0 all OpenStack identity service - Common files
ii python-keystoneauth1 3.13.1-0ubuntu1~cloud0 all authentication library for OpenStack Identity - Python 2.7
ii python-keystoneclient 1:3.19.0-0ubuntu1~cloud0 all client library for the OpenStack Keystone API - Python 2.x
ii python-keystonemiddleware 6.0.0-0ubuntu1~cloud0 all Middleware for OpenStack Identity (Keystone) - Python 2.x
ii python3-keystone 2:16.0.0-0ubuntu1~cloud0 all OpenStack identity service - Python 3 library
ii python3-keystoneauth1 3.17.1-0ubuntu1~cloud0 all authentication library for OpenStack Identity - Python 3.x
ii python3-keystoneclient 1:3.21.0-0ubuntu1~cloud0 all client library for the OpenStack Keystone API - Python 3.x
ii python3-keystonemiddleware 7.0.1-0ubuntu1~cloud0 all Middleware for OpenStack Identity (Keystone) - Python 3.x

Stein/RHEL:
$ rpm -qa | grep keystone
python3-keystoneclient-3.19.0-0.20190312070330.6c4bb8b.el8ost.noarch
openstack-keystone-15.0.1-0.20190720060412.5f27c4b.el8ost.noarch
python3-keystoneauth1-3.13.1-0.20190311052414.bde07bc.el8ost.noarch
python3-keystonemiddleware-6.0.0-0.20190312071144.fca37ea.el8ost.noarch
python3-keystone-15.0.1-0.20190720060412.5f27c4b.el8ost.noarch

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[Colleen Murphy (krinkle)]

Partial patch (needs a test)

[Colleen Murphy (krinkle)]

Full patch attached

[Jeremy Stanley (fungi)]

Since the details were already disclosed[0] earlier today in #openstack-keystone, I recommend we don't maintain an embargo for this and instead treat it as a public report[1].

[0] http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-keystone.2019-12-04.log.html#t2019-12-04T09:49:13
[1] https://security.openstack.org/vmt-process.html#process

[Daniel 'f0o' Preussker (dpreussker)]

Apologies for the IRC disclosure, this was before I realized what I just ran into. I fully suspected the issue within my policy definition and only upon further looking at it and cross-referencing on a different OpenStack platform I understood that this was indeed a Security Issue and not just a config fail.

Should I request a CVE on your behalf? Or is that being tracked by the VulnTeam?

[Gage Hugo (gagehugo)]

You can request a CVE if you wish, but the VMT usually does so as part of the process[0]. So either way one will get made for this.

[0] https://security.openstack.org/vmt-process.html#send-cve-request

[Daniel 'f0o' Preussker (dpreussker)]

If the VMT wants to take it with Mitre I'm more than happy with that. Last time I was involved with Octavia where they didnt had VMT correspondence so I filled out the forms for them.

Nice to be able to lean back for once as a reporter, just kidding ;)

Please do let me know if I can be of any help or assist in any way.

I can apply and test patches against Train/Ubuntu (UCA repo) but sadly not against RHEL because that's running Red Hat OpenStack Platform (support would void).

[Gage Hugo (gagehugo)]

Agreed with fungi, with the details of this out in IRC, we can treat this as a public report. Looks like we have a patch already as well, so things are moving fairly quickly in terms of getting this fixed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/697355

[Jeremy Stanley (fungi)]

The OpenStack VMT will request a CVE assignment from MITRE once we agree on a complete impact description for this report. If you're interested in the details of our report handling processes, you can find them here: https://security.openstack.org/vmt-process.html#process

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "1855080.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Gage Hugo (gagehugo)]

I wrote up an impact description for use in an upcoming OpenStack Security Advisory and associated CVE request. Please suggest any improvements or suggestions:

Title: Credentials API allows listing and retrieving of all user's credentials
Reporter: Daniel 'f0o' Preussker ()
Products: Keystone
Affects: >=15.0.0, <=16.0.0

Description:
Daniel 'f0o' Preussker reported a vulnerability in Keystone's list credentials API. Any user with a role on a project is able to list any credentials with the /v3/credentials API when enforce_scope is false. Users with a role on a project are able to view any other users' credentials, which could leak sign-on information for Time-based One Time Passwords (TOTP) or othewise. Deployments running keystone with enforce_scope set to false are affected.

[Jeremy Stanley (fungi)]

Daniel, is there any organization you want credited along with you for reporting this defect?

Gage, I think the use of "user's" in the title (copied from the report itself) incorrectly suggests that a user only has access to credentials for their own user rather than, as the description explains, for all users in that project. Instead maybe try "Credentials API allows listing and retrieving of project credentials" or something like that? As for the affects line, assuming this problem was only introduced in Stein, you want "==15.0.0, ==16.0.0" (wow, were there really no stable/stein point releases?!?) or alternatively ">=15.0.0 <15.0.1, >=16.0.0 <16.0.1" to accurately reflect that any point releases will contain the fix.

[Daniel 'f0o' Preussker (dpreussker)]

No. I'm not affiliated with any organisation in this regard. Just like the Octavia OSSA haha.

Thanks for asking tho :)

On December 5, 2019 5:57:21 PM UTC, Jeremy Stanley <email address hidden> wrote:
>Daniel, is there any organization you want credited along with you for
>reporting this defect?
>
>Gage, I think the use of "user's" in the title (copied from the report
>itself) incorrectly suggests that a user only has access to credentials
>for their own user rather than, as the description explains, for all
>users in that project. Instead maybe try "Credentials API allows
>listing
>and retrieving of project credentials" or something like that? As for
>the affects line, assuming this problem was only introduced in Stein,
>you want "==15.0.0, ==16.0.0" (wow, were there really no stable/stein
>point releases?!?) or alternatively ">=15.0.0 <15.0.1, >=16.0.0
><16.0.1"
>to accurately reflect that any point releases will contain the fix.
>
>--
>You received this bug notification because you are subscribed to the
>bug
>report.
>https://bugs.launchpad.net/bugs/1855080
>
>Title:
> Credentials API allows listing and retrieving of all user's
> credentials
>
>Status in OpenStack Identity (keystone):
> In Progress
>Status in OpenStack Security Advisory:
> Confirmed
>Status in keystone package in Ubuntu:
> New
>
>Bug description:
> Tested against Stein and Train.
>
> # User creating a credential, i.e totp or similar
> $ OS_CLOUD=1 openstack token issue
> | project_id | c3caf1b55bb84b78a795fd81838e5160
> | user_id | 9971b0f13d2d4a578212d028a53c3209
>$ OS_CLOUD=1 openstack credential create --type test
>9971b0f13d2d4a578212d028a53c3209 test-data
> $ OS_CLOUD=1 openstack credential list
>+----------------------------------+------+----------------------------------+-----------+------------+
>| ID | Type | User ID
> | Data | Project ID |
>+----------------------------------+------+----------------------------------+-----------+------------+
>| 0a3a2d3b7dad4886b0bbf61b6cd7d2b0 | test |
>9971b0f13d2d4a578212d028a53c3209 | test-data | None |
>+----------------------------------+------+----------------------------------+-----------+------------+
>
> # Different User but same Project
> $ OS_CLOUD=2 openstack token issue
> | project_id | c3caf1b55bb84b78a795fd81838e5160
> | user_id | 6b28a0b073fc4ac7843f33190ebc5c3c
> $ OS_CLOUD=2 openstack credential list
>+----------------------------------+------+----------------------------------+-----------+------------+
>| ID | Type | User ID
> | Data | Project ID |
>+----------------------------------+------+----------------------------------+-----------+------------+
>| 0a3a2d3b7dad4886b0bbf61b6cd7d2b0 | test |
>9971b0f13d2d4a578212d028a53c3209 | test-data | None |
>+----------------------------------+------+----------------------------------+-----------+------------+
>
> # Different User and Different Project
> $ OS_CLOUD=3 openstack token issue
> | project_id | d43f20ae5a7e4f36b701710277384401
> | user_id | 2e48f1a7d1474391a826a2b9700e5949
> $ OS_CLOUD=3 openstack cre...

[Gage Hugo (gagehugo)]

Updated, please review:

Title: Credentials API allows non-admin to list and retrieve every users' credentials
Reporter: Daniel 'f0o' Preussker
Products: Keystone
Affects: ==15.0.0, ==16.0.0

Description:
Daniel 'f0o' Preussker reported a vulnerability in Keystone's list credentials API. Any user with a role on a project is able to list any credentials with the /v3/credentials API when enforce_scope is false. Users with a role on a project are able to view any other users' credentials, which could leak sign-on information for Time-based One Time Passwords (TOTP) or othewise. Deployments running keystone with enforce_scope set to false are affected. There will be a slight performance impact for the list credentials API once this issue is fixed.

[Jeremy Stanley (fungi)]

Somewhat of a grammar nit on the updated title, but it would be "every user's" or "all users'" (placement of the apostrophe in possessive nouns is significant for indicating plurality, and "every" modifies a singular noun as opposed to "all" which modifies a plural). This nuance in the English language is why I suggested dodging "users'" or "user's" and picking different, less ambiguous phrasing so as to avoid confusion for non-native readers of English when skimming the advisory title.

[Gage Hugo (gagehugo)]

Ah ok, I'll remove the apostrophe then.

Updated, please review:

Title: Credentials API allows non-admin to list and retrieve all users credentials
Reporter: Daniel 'f0o' Preussker
Products: Keystone
Affects: ==15.0.0, ==16.0.0

Description:
Daniel 'f0o' Preussker reported a vulnerability in Keystone's list credentials API. Any user with a role on a project is able to list any credentials with the /v3/credentials API when enforce_scope is false. Users with a role on a project are able to view any other users' credentials, which could leak sign-on information for Time-based One Time Passwords (TOTP) or othewise. Deployments running keystone with enforce_scope set to false are affected. There will be a slight performance impact for the list credentials API once this issue is fixed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/697355
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=17c337dbdbfb9d548ad531c2ad0483c9bce5b98f
Submitter: Zuul
Branch: master

commit 17c337dbdbfb9d548ad531c2ad0483c9bce5b98f
Author: Colleen Murphy <email address hidden>
Date: Wed Dec 4 10:51:05 2019 -0800

    Fix credential list for project members

    Without this patch, project members and readers can list any credentials
    with the /v3/credentials API when enforce_scope is false. enforce_scope
    is only applicable to project admins due to the admin-ness problem[1],
    and this policy is not meant to allow project admins any access to users'
    credentials (only system admins should be able to access them). However,
    when enforce_scope is false, we need to preserve the old behavior of
    project admins being able to list all credentials. This change mitigates
    the problem by running the identity:get_credential policy check to
    filter out credentials the user does not have access to. This will
    impact performance.

    Closes-bug: #1855080

    [1] https://bugs.launchpad.net/keystone/+bug/968696

    Change-Id: I5dd85a6b8368373a27aef2942a64499d020662ef

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/697611

[Jeremy Stanley (fungi)]

Just to get confirmation, this bug was only introduced as of Stein, right? It's not present in Rocky or earlier?

Gage, assuming the above is true, and if nobody has any other concerns about your proposed impact description in comment #17, you can probably go ahead and request a CVE assignment for this so we can proceed with the advisory, since the fix has already merged to master and it looks like stable backports are in the process of getting proposed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/697611
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bd3f63787151183f4daa43578aa491856fefae5b
Submitter: Zuul
Branch: stable/train

commit bd3f63787151183f4daa43578aa491856fefae5b
Author: Colleen Murphy <email address hidden>
Date: Wed Dec 4 10:51:05 2019 -0800

    Fix credential list for project members

    Without this patch, project members and readers can list any credentials
    with the /v3/credentials API when enforce_scope is false. enforce_scope
    is only applicable to project admins due to the admin-ness problem[1],
    and this policy is not meant to allow project admins any access to users'
    credentials (only system admins should be able to access them). However,
    when enforce_scope is false, we need to preserve the old behavior of
    project admins being able to list all credentials. This change mitigates
    the problem by running the identity:get_credential policy check to
    filter out credentials the user does not have access to. This will
    impact performance.

    Closes-bug: #1855080

    [1] https://bugs.launchpad.net/keystone/+bug/968696

    Change-Id: I5dd85a6b8368373a27aef2942a64499d020662ef
    (cherry picked from commit 17c337dbdbfb9d548ad531c2ad0483c9bce5b98f)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/697731

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/697731
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=17947516b0095c51da5cff94771247f2e7c44ee6
Submitter: Zuul
Branch: stable/stein

commit 17947516b0095c51da5cff94771247f2e7c44ee6
Author: Colleen Murphy <email address hidden>
Date: Wed Dec 4 10:51:05 2019 -0800

    Fix credential list for project members

    Without this patch, project members and readers can list any credentials
    with the /v3/credentials API when enforce_scope is false. enforce_scope
    is only applicable to project admins due to the admin-ness problem[1],
    and this policy is not meant to allow project admins any access to users'
    credentials (only system admins should be able to access them). However,
    when enforce_scope is false, we need to preserve the old behavior of
    project admins being able to list all credentials. This change mitigates
    the problem by running the identity:get_credential policy check to
    filter out credentials the user does not have access to. This will
    impact performance.

    Closes-bug: #1855080

    [1] https://bugs.launchpad.net/keystone/+bug/968696

    Change-Id: I5dd85a6b8368373a27aef2942a64499d020662ef
    (cherry picked from commit 17c337dbdbfb9d548ad531c2ad0483c9bce5b98f)
    (cherry picked from commit bd3f63787151183f4daa43578aa491856fefae5b)

[Daniel 'f0o' Preussker (dpreussker)]

I honestly don't know if it's been in Rocky or not. The code change suggests that it got introduced with the system scoping which appeared in Stein as far as I know.

[Gage Hugo (gagehugo)]

I wasn't able to recreate this with Rocky, only a user with the "admin" role was able to list credentials, other users with member roles were denied (as policy defined).

The code was indeed changed after Rocky to account for system scope, where I believe that this issue was introduced.

[Gage Hugo (gagehugo)]

OSSA Report: https://review.opendev.org/#/c/698045/

[Alex Murray (alexmurray)]

This was assigned CVE-2019-19687 and was fixed via https://usn.ubuntu.com/4262-1/ in Ubuntu.