Activity log for bug #1260080

Date Who What changed Old value New value Message
2013-12-11 21:47:24 Morgan Fainberg bug added bug
2013-12-11 21:47:39 Morgan Fainberg bug added subscriber Adam Young
2013-12-11 21:47:50 Morgan Fainberg bug added subscriber Dolph Mathews
2013-12-11 21:48:18 Morgan Fainberg bug added subscriber Steven Hardy
2013-12-11 21:48:43 Morgan Fainberg nominated for series keystone/grizzly
2013-12-11 21:48:43 Morgan Fainberg bug task added keystone/grizzly
2013-12-11 21:48:43 Morgan Fainberg nominated for series keystone/havana
2013-12-11 21:48:43 Morgan Fainberg bug task added keystone/havana
2013-12-11 22:39:59 Morgan Fainberg description When using the memcache token backend, In cases where token revocations (bulk revocations) that rely on the trustee's token-index-list to perform the revocations - the revocations cannot occur. This is because trustee tokens are added only to the trustor's token list. This appears to only be an issue for trusts with impersonation enabled. This is most noticeable when the trustee user is disabled or the trustee changes a password. I am sure there are other related scenarios. Example reproduction - Create the trust: curl -H "X-Auth-Token: $TOKEN" -X POST http://127.0.0.1:35357/v3/OS-TRUST/trusts -d '{"trust":{"impersonation": true, "project_id": "9b2c18a72c8a477b891b48931c26cebe", "roles": [{"name":"admin"}],"trustee_user_id":"36240e5f6ed04857bf15d08a519b6e37","trustor_user_id":"46c9e36c50c14dfab36b6511d6ebedd4"}}' -H 'Content-Type: application/json' RESPONSE: { "trust": { "expires_at": null, "id": "0d2dc361043d4f9d8a84a7f253e20924", "impersonation": true, "links": { "self": "http://172.16.30.195:5000/v3/OS-TRUST/trusts/0d2dc361043d4f9d8a84a7f253e20924" }, "project_id": "9b2c18a72c8a477b891b48931c26cebe", "roles": [ { "id": "0bd1d61badd742bebc044dc246a43513", "links": { "self": "http://172.16.30.195:5000/v3/roles/0bd1d61badd742bebc044dc246a43513" }, "name": "admin" } ], "roles_links": { "next": null, "previous": null, "self": "http://172.16.30.195:5000/v3/OS-TRUST/trusts/0d2dc361043d4f9d8a84a7f253e20924/roles" }, "trustee_user_id": "36240e5f6ed04857bf15d08a519b6e37", "trustor_user_id": "46c9e36c50c14dfab36b6511d6ebedd4" } } Consume the trust: vagrant@precise64:~/devstack$ cat trust_token.json { "auth": { "identity": { "methods": [ "token" ], "token": { "id": "<PKI TOKEN ID>" } }, "scope": { "OS-TRUST:trust": { "id": "0d2dc361043d4f9d8a84a7f253e20924" } } } } curl -si -d @trust_token.json -H 'Content-Type: application/json' -X POST http://localhost:35357/v3/auth/tokens RESPONSE: { "token": { "OS-TRUST:trust": { "id": "0d2dc361043d4f9d8a84a7f253e20924", "impersonation": true, "trustee_user": { "id": "36240e5f6ed04857bf15d08a519b6e37" }, "trustor_user": { "id": "46c9e36c50c14dfab36b6511d6ebedd4" } }, "catalog": [ ... <SNIP FOR BREVITY> ], "expires_at": "2013-12-12T20:06:00.239812Z", "extras": {}, "issued_at": "2013-12-11T20:10:22.224381Z", "methods": [ "token", "password" ], "project": { "domain": { "id": "default", "name": "Default" }, "id": "9b2c18a72c8a477b891b48931c26cebe", "name": "admin" }, "roles": [ { "id": "0bd1d61badd742bebc044dc246a43513", "name": "admin" } ], "user": { "domain": { "id": "default", "name": "Default" }, "id": "46c9e36c50c14dfab36b6511d6ebedd4", "name": "admin" } } } Check the memcache token lists, the admin user should have 1 token (used to create the trust), the demo user should have 2 tokens (initial token, and trust token) vagrant@precise64:~/devstack$ telnet localhost 11211 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. get usertokens-46c9e36c50c14dfab36b6511d6ebedd4 VALUE usertokens-46c9e36c50c14dfab36b6511d6ebedd4 0 104 "1bb6a8abc42e1d1a944e08a20de20a91","ee512379a66027733001648799083349" END get usertokens-36240e5f6ed04857bf15d08a519b6e37 VALUE usertokens-36240e5f6ed04857bf15d08a519b6e37 0 34 "36df008ec5b5d3dd6983c8fe84d407f6" This does not affect the SQL or KVS backends. This will affect Master (until KVS refactor is complete), Havana, and Grizzly. The reason for this occurring is a lack of testing against the memcache backend itself (and it’s logic). With the refactor of KVS (keyvaluestore) it will ensure the testing is the same against all KVS (memcache, in-memory, redis, etc) backends since the interfaces work in the same manner/mechanism. The root of this issue stems from the old KVS and SQL systems being less strict on matching trust scoped tokens: (KVS) http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/kvs.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n86 (SQL) http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/sql.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n126 Whereas memcache needs to use the user-index to determine tokens for a given user before matching against the trust scope: http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/memcache.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n190 When using the memcache token backend, In cases where token revocations (bulk revocations) that rely on the trustee's token-index-list to perform the revocations - the revocations cannot occur. This is because trustee tokens are added only to the trustor's token list. This appears to only be an issue for trusts with impersonation enabled. This is most noticeable when the trustee user is disabled or the trustee changes a password. I am sure there are other related scenarios. Example reproduction - Create the trust: curl -H "X-Auth-Token: $TOKEN" -X POST http://127.0.0.1:35357/v3/OS-TRUST/trusts -d '{"trust":{"impersonation": true, "project_id": "9b2c18a72c8a477b891b48931c26cebe", "roles": [{"name":"admin"}],"trustee_user_id":"36240e5f6ed04857bf15d08a519b6e37","trustor_user_id":"46c9e36c50c14dfab36b6511d6ebedd4"}}' -H 'Content-Type: application/json' RESPONSE: {     "trust": {         "expires_at": null,         "id": "0d2dc361043d4f9d8a84a7f253e20924",         "impersonation": true,         "links": {             "self": "http://172.16.30.195:5000/v3/OS-TRUST/trusts/0d2dc361043d4f9d8a84a7f253e20924"         },         "project_id": "9b2c18a72c8a477b891b48931c26cebe",         "roles": [             {                 "id": "0bd1d61badd742bebc044dc246a43513",                 "links": {                     "self": "http://172.16.30.195:5000/v3/roles/0bd1d61badd742bebc044dc246a43513"                 },                 "name": "admin"             }         ],         "roles_links": {             "next": null,             "previous": null,             "self": "http://172.16.30.195:5000/v3/OS-TRUST/trusts/0d2dc361043d4f9d8a84a7f253e20924/roles"         },         "trustee_user_id": "36240e5f6ed04857bf15d08a519b6e37",         "trustor_user_id": "46c9e36c50c14dfab36b6511d6ebedd4"     } } Consume the trust: vagrant@precise64:~/devstack$ cat trust_token.json {     "auth": {         "identity": {             "methods": [                 "token"             ],             "token": {                 "id": "<PKI TOKEN ID>"             }         },         "scope": {             "OS-TRUST:trust": {                 "id": "0d2dc361043d4f9d8a84a7f253e20924"             }         }     } } curl -si -d @trust_token.json -H 'Content-Type: application/json' -X POST http://localhost:35357/v3/auth/tokens RESPONSE: {     "token": {         "OS-TRUST:trust": {             "id": "0d2dc361043d4f9d8a84a7f253e20924",             "impersonation": true,             "trustee_user": {                 "id": "36240e5f6ed04857bf15d08a519b6e37"             },             "trustor_user": {                 "id": "46c9e36c50c14dfab36b6511d6ebedd4"             }         },         "catalog": [            ... <SNIP FOR BREVITY>         ],         "expires_at": "2013-12-12T20:06:00.239812Z",         "extras": {},         "issued_at": "2013-12-11T20:10:22.224381Z",         "methods": [             "token",             "password"         ],         "project": {             "domain": {                 "id": "default",                 "name": "Default"             },             "id": "9b2c18a72c8a477b891b48931c26cebe",             "name": "admin"         },         "roles": [             {                 "id": "0bd1d61badd742bebc044dc246a43513",                 "name": "admin"             }         ],         "user": {             "domain": {                 "id": "default",                 "name": "Default"             },             "id": "46c9e36c50c14dfab36b6511d6ebedd4",             "name": "admin"         }     } } Check the memcache token lists, the admin user should have 1 token (used to create the trust), the demo user should have 2 tokens (initial token, and trust token) ADMIN-ID: 46c9e36c50c14dfab36b6511d6ebedd4 DEMO-ID: 36240e5f6ed04857bf15d08a519b6e37 vagrant@precise64:~/devstack$ telnet localhost 11211 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. get usertokens-46c9e36c50c14dfab36b6511d6ebedd4 VALUE usertokens-46c9e36c50c14dfab36b6511d6ebedd4 0 104 "1bb6a8abc42e1d1a944e08a20de20a91","ee512379a66027733001648799083349" END get usertokens-36240e5f6ed04857bf15d08a519b6e37 VALUE usertokens-36240e5f6ed04857bf15d08a519b6e37 0 34 "36df008ec5b5d3dd6983c8fe84d407f6" This does not affect the SQL or KVS backends. This will affect Master (until KVS refactor is complete), Havana, and Grizzly. The reason for this occurring is a lack of testing against the memcache backend itself (and it’s logic). With the refactor of KVS (keyvaluestore) it will ensure the testing is the same against all KVS (memcache, in-memory, redis, etc) backends since the interfaces work in the same manner/mechanism. The root of this issue stems from the old KVS and SQL systems being less strict on matching trust scoped tokens: (KVS) http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/kvs.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n86 (SQL) http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/sql.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n126 Whereas memcache needs to use the user-index to determine tokens for a given user before matching against the trust scope: http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/backends/memcache.py?id=cc0c6565e3fe6ed60543e6ebd5e54cba51e59fc6#n190
2013-12-11 23:55:38 Jeremy Stanley bug task added ossa
2013-12-13 13:44:00 Thierry Carrez ossa: status New Incomplete
2013-12-13 21:18:56 Adam Young ossa: status Incomplete Confirmed
2013-12-16 16:51:47 Thierry Carrez ossa: importance Undecided High
2014-01-06 13:30:12 Thierry Carrez information type Private Security Public Security
2014-01-21 16:52:03 Tristan Cacqueray ossa: assignee Tristan Cacqueray (tristan-cacqueray)
2014-01-24 19:06:08 Thierry Carrez ossa: status Confirmed Triaged
2014-02-17 22:49:44 Dolph Mathews keystone: status New In Progress
2014-02-17 22:49:56 Dolph Mathews keystone: assignee Morgan Fainberg (mdrnstm)
2014-02-17 22:49:59 Dolph Mathews keystone: importance Undecided Medium
2014-02-17 23:30:01 Morgan Fainberg keystone/grizzly: importance Undecided Medium
2014-02-17 23:30:04 Morgan Fainberg keystone/havana: importance Undecided Medium
2014-02-17 23:30:15 Morgan Fainberg keystone: milestone icehouse-3
2014-02-20 04:46:42 OpenStack Infra keystone: status In Progress Fix Committed
2014-02-21 19:18:53 Morgan Fainberg keystone/grizzly: assignee Morgan Fainberg (mdrnstm)
2014-02-21 19:18:56 Morgan Fainberg keystone/havana: assignee Morgan Fainberg (mdrnstm)
2014-02-21 19:18:59 Morgan Fainberg keystone/grizzly: status New In Progress
2014-02-21 19:19:01 Morgan Fainberg keystone/havana: status New In Progress
2014-03-03 09:43:22 Tristan Cacqueray summary Trustee token revocations with memcache backend Trustee token revocations with memcache backend (CVE-2014-2237)
2014-03-03 13:40:53 Colin Watson cve linked 2014-2237
2014-03-04 16:39:50 Tristan Cacqueray ossa: status Triaged Fix Committed
2014-03-05 19:54:55 Thierry Carrez keystone: status Fix Committed Fix Released
2014-03-06 02:23:47 OpenStack Infra keystone/grizzly: status In Progress Fix Committed
2014-03-06 02:26:57 OpenStack Infra keystone/havana: status In Progress Fix Committed
2014-03-06 11:06:06 Thierry Carrez ossa: status Fix Committed Fix Released
2014-03-20 19:55:43 Alan Pevec keystone/grizzly: milestone 2013.1.5
2014-03-20 19:56:59 Alan Pevec summary Trustee token revocations with memcache backend (CVE-2014-2237) [OSSA 2014-006] Trustee token revocations with memcache backend (CVE-2014-2237)
2014-03-20 22:12:24 Alan Pevec keystone/grizzly: status Fix Committed Fix Released
2014-03-28 20:52:38 Adam Gandelman keystone/havana: milestone 2013.2.3
2014-04-03 19:21:26 Adam Gandelman keystone/havana: status Fix Committed Fix Released
2014-04-17 08:00:22 Thierry Carrez keystone: milestone icehouse-3 2014.1