[OSSA-2012-018] Removing user from a tenant isn't invalidating user access to tenant

[Vijaya Erukala]

Was: (Able to access ec2 resources with out a user-role)

Steps to reproduce:

1.I have created a user,tenant and ec2 credentials using keystone and didnt associated a role
then try to run
$ euca-describe-images
Unauthorized: Failure communicating with keystone2

2.then associated a role to the user
$keystone user-role-add --user-id ee32bff3e1524a2b82fe804aac0ce682 --tenant-id cf38a72df8a14bb8984c699edfceb1c3 --role-id 26760c0f9ed045fcb70112f7b16813b3

3.then again I ran
$ euca-describe-images
IMAGE ami-00000002 None (cirros-0.3.0-x86_64-uec) 597b866b450d491f889b9432a564e9a9 available public machineaki-00000001 ari-00000003 instance-store
IMAGE ari-00000003 None (cirros-0.3.0-x86_64-uec-ramdisk) 597b866b450d491f889b9432a564e9a9 available public ramdisk instance-store
IMAGE aki-00000001 None (cirros-0.3.0-x86_64-uec-kernel) 597b866b450d491f889b9432a564e9a9 available public kernel instance-store

4.remove the user-role
keystone user-role-delete --user-id ee32bff3e1524a2b82fe804aac0ce682 --tenant-id cf38a72df8a14bb8984c699edfceb1c3 --role-id 26760c0f9ed045fcb70112f7b16813b3

5.now the user doesn't have any role associated,now again run
 $ euca-describe-images
IMAGE ami-00000002 None (cirros-0.3.0-x86_64-uec) 597b866b450d491f889b9432a564e9a9 available public machineaki-00000001 ari-00000003 instance-store
IMAGE ari-00000003 None (cirros-0.3.0-x86_64-uec-ramdisk) 597b866b450d491f889b9432a564e9a9 available public ramdisk instance-store
IMAGE aki-00000001 None (cirros-0.3.0-x86_64-uec-kernel) 597b866b450d491f889b9432a564e9a9 available public kernel instance-store

here user should get the message as in step 2 but it was successful.

not sure if this is a bug with keystone or ec2.

[Russell Bryant (russellb)]

I have subscribed the vulnerability management team to this bug. We need to evaluate this as a potential security vulnerability.

[Thierry Carrez (ttx)]

If confirmed, it's in the "extension of validity" class like the ones where you could extend the lifetime of a token. A bit hard to exploit (since you need to be given the role in the first place) but definitely needs to be fixed. I would issue an OSSA on this for the same reason we issued one on the previous token issues.

[Joseph Heck (heckj)]

The EC2 mechanisms disregard role entirely, and work from the relationship of the user to the tenant. If that relationship didn't exist, the user shouldn't be able to access the tenant's resources through EC2. WIth the V2 API, just associating a user with a tenant is an implicit "membership" role - and in fact that's how Horizon displays this information.

Actual "roles" as Keystone defines them are irrelevant to the EC2 credential access, and with the V2 API's and the EC2 mechanisms, roles have no function there. They're not checked as a form of RBAC for allowing EC2 credential access.

The V3 API is aiming to clean this up a bit, more for the OpenStack API. V3 API changes does not currently specify any RBAC around EC2 credential authority.

[Vish Ishaya (vishvananda)]

We don't check that the user is still a member of the tenant (project) so the ec2 (or s3) token still works

[Joseph Heck (heckj)]

vishy: heckj: correct, we don't check anywhere that the user is still a member of the tenant
[2:29pm] heckj: vishy: so the core of that bug is that we're not verifying that the user is a member of the tenant?
[2:30pm] vishy: correct
[2:30pm] vishy: i just marked it confirmed
[2:30pm] heckj: heh - I just marked it as triaged
[2:30pm] vishy: i just tested and removed myself from my tenant, I was still able to run commands with my ec2 token
[2:30pm] heckj: Okay - thank you! I'll get on it!

DELETE /tenants/{tenant_id}/users/{user_id}/roles/OS-KSADM/{role_id}
in keystone/contrib/admin_crud/core.py

[Joseph Heck (heckj)]

[2:31pm] vishy: i can still do stuff through the nova api as well!

[Joseph Heck (heckj)]

related bug around removing a user from a tenant:
https://bugs.launchpad.net/python-keystoneclient/+bug/1078497

[Vish Ishaya (vishvananda)]

ok tracked this down and verified. The other bug above is separate, but this one still occurs. I managed to successfuly delete the role and the ec2 commands still work. This is because in the ec2 credentials we are storing the user and tenant. We should fix this by adding a check in the ec2 authenticate to make sure that the user still has a valid role before validating.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/16028

[Thierry Carrez (ttx)]

Looks like security is back on track !

[Thierry Carrez (ttx)]

So if I get this straight, the issue is that if a user has a role that makes it associated with a tenant, but then that role is removed, the token is still valid. That would make it part of the "extension of validity" class I mentioned above, which we considered in the past as worthy of an advisory.

If confirmed, I'll prepare an impact statement.

Vish's folsom-backport tag probably means that Folsom is also affected. What about Essex at first glance ?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/16028
Committed: http://github.com/openstack/keystone/commit/9d68b40cb9ea818c48152e6c712ff41586ad9653
Submitter: Jenkins
Branch: master

commit 9d68b40cb9ea818c48152e6c712ff41586ad9653
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16304

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/16304
Committed: http://github.com/openstack/keystone/commit/37308dd4f3e33f7bd0f71d83fd51734d1870713b
Submitter: Jenkins
Branch: stable/folsom

commit 37308dd4f3e33f7bd0f71d83fd51734d1870713b
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

[Thierry Carrez (ttx)]

@Russell: we have a precedent for issuing after-the-fact OSSA on this kind of thing. Do you think we should here ? If yes, maybe we can give a few days warning to the stakeholders while we work on the impact statement ?

[Russell Bryant (russellb)]

Yes, I think based on our precedent, we should issue an OSSA for this.

[Thierry Carrez (ttx)]

Would be good to push an Essex backport for this as well -- CCing stable/essex team

Proposed impact description:

Title: EC2-style credentials invalidation issue
Reporter: Vijaya Erukala
Products: Keystone
Affects: All versions

Description:
Vijaya Erukala reported a vulnerability in Keystone EC2-style credentials invalidation: when a user is removed from a tenant, issued EC2-style credentials would continue to be valid for that tenant. An authenticated and authorized user could potentially leverage this vulnerability to extend his access beyond the account owner expectations. Only setups enabling EC2-style credentials (for example enabling EC2 API in Nova) are affected.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/16759

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/16760

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/16760
Committed: http://github.com/openstack/keystone/commit/8735009dc5b895db265a1cd573f39f4acfca2a19
Submitter: Jenkins
Branch: stable/essex

commit 8735009dc5b895db265a1cd573f39f4acfca2a19
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

[Russell Bryant (russellb)]

@ttx: The vulnerability description sounds good to me. Thanks!

[Thierry Carrez (ttx)]

Pushed to downstream stakeholders. CVE-2012-5571

[Thierry Carrez (ttx)]

OSSA 2012-018

[Jamie Strandboge (jdstrand)]

This was fixed in http://www.ubuntu.com/usn/usn-1641-1/.

[Clint Byrum (clint-fewbar)]

Hello Vijaya, or anyone else affected,

Accepted keystone into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/keystone/2012.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Vijaya Erukala (vijaya-erukala)]

I have verified this on grizzly-2 and validation seems to be working fine.

$ keystone user-role-remove --user bd9c28b6e2794574b030ad5c3a7e4818 --tenant 97c5778a10f34b499a5abe1e2ecff24c --role 9fe2ff9ee4384b1894a90878d3e92bab

$ euca-describe-instances
Unauthorized: Failure communicating with keystone

[Brian Murray (brian-murray)]

Hello Vijaya, or anyone else affected,

Accepted keystone into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/keystone/2012.1.3+stable-20130423-f48dd0fc-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Yolanda Robla (yolanda.robla)]

Please find the attached test log from the Ubuntu Server Team's CI infrastructure. As part of the verification process for this bug, Keystone has been deployed and configured across multiple nodes using precise-proposed as an installation source. After successful bring-up and configuration of the cluster, a number of exercises and smoke tests have be invoked to ensure the updated package did not introduce any regressions. A number of test iterations were carried out to catch any possible transient errors.

Please Note the list of installed packages at the top and bottom of the report.

For records of upstream test coverage of this update, please see the Jenkins links in the comments of the relevant upstream code-review(s):

Trunk review: https://review.openstack.org/16028
Stable review: https://review.openstack.org/16760

As per the provisional Micro Release Exception granted to this package by the Technical Board, we hope this contributes toward verification of this update.

[Yolanda Robla (yolanda.robla)]

Test coverage log.

[Scott Kitterman (kitterman)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.