| 2016-05-17 07:57:44 |
jackning |
bug |
|
|
added bug |
| 2016-05-17 07:58:00 |
jackning |
keystone: assignee |
|
jackning (ningyougang) |
|
| 2016-05-17 07:59:54 |
jackning |
description |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.
And second time query,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
the query costs 7 seconds,but first query almost costs 50 seconds |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.
And second time query,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
the query costs 7 seconds,but first query almost costs 50 seconds
So i think it is very nessary to improve this feature |
|
| 2016-05-17 08:03:39 |
jackning |
description |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.
And second time query,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
the query costs 7 seconds,but first query almost costs 50 seconds
So i think it is very nessary to improve this feature |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.
And second time query,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
|
| 2016-05-17 08:42:37 |
OpenStack Infra |
keystone: status |
New |
In Progress |
|
| 2016-05-17 08:43:56 |
jackning |
description |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.
And second time query,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second time query leter,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
|
| 2016-05-17 08:44:50 |
jackning |
description |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second time query leter,it also goes to db,not use the cache to improve the speed.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time leter,it also goes to db,not use the cache.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
|
| 2016-05-17 16:42:21 |
Thomas Hsiao |
bug |
|
|
added subscriber Thomas Hsiao |
| 2016-05-18 01:59:46 |
jackning |
description |
In our project,we found that the speed of query user from ldap server was very slow(our ldap user number is 12,000,the query costs almost 45 seconds)
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time leter,it also goes to db,not use the cache.
So we can add the cache to improve the query speed
After add @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580,
first query time almost costs 50 seconds,but second time leter only costs 7 seconds.
So i think it is very nessary to improve this feature |
In our project, the speed of query user from ldap server is very slow,our ldap user number is 12,000,the query costs almost 45 seconds
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time later,it also goes to db,not use the cache.
So adding the cache to improve the query speed
After adding @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580.
First query time almost costs 50 seconds,but second query time later it only costs 7 seconds.
So it is very necessary to improve this feature |
|
| 2016-06-09 00:38:09 |
Samuel de Medeiros Queiroz |
keystone: importance |
Undecided |
Wishlist |
|
| 2016-06-13 03:06:32 |
OpenStack Infra |
keystone: assignee |
jackning (ningyougang) |
Andrew Liu (andrew-lhj) |
|
| 2016-07-08 01:04:00 |
OpenStack Infra |
keystone: assignee |
Andrew Liu (andrew-lhj) |
Boris Bobrov (bbobrov) |
|
| 2016-07-11 01:46:38 |
OpenStack Infra |
keystone: assignee |
Boris Bobrov (bbobrov) |
Andrew Liu (andrew-lhj) |
|
| 2016-07-11 12:48:52 |
Lars Erik Pedersen |
bug |
|
|
added subscriber Lars Erik Pedersen |
| 2016-07-15 12:07:21 |
Alexander Petrov |
bug |
|
|
added subscriber Alexander Petrov |
| 2016-07-15 18:38:09 |
OpenStack Infra |
keystone: assignee |
Andrew Liu (andrew-lhj) |
Boris Bobrov (bbobrov) |
|
| 2016-07-18 03:28:50 |
OpenStack Infra |
keystone: assignee |
Boris Bobrov (bbobrov) |
Andrew Liu (andrew-lhj) |
|
| 2016-07-18 09:48:39 |
OpenStack Infra |
keystone: assignee |
Andrew Liu (andrew-lhj) |
Boris Bobrov (bbobrov) |
|
| 2016-07-20 03:00:19 |
OpenStack Infra |
keystone: assignee |
Boris Bobrov (bbobrov) |
Andrew Liu (andrew-lhj) |
|
| 2016-07-25 02:59:26 |
OpenStack Infra |
keystone: status |
In Progress |
Fix Released |
|
| 2016-07-25 12:12:15 |
Steve Martinelli |
keystone: milestone |
|
newton-3 |
|
| 2016-08-31 12:14:32 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto Murata |
| 2018-01-10 01:10:04 |
Billy Olsen |
bug task added |
|
keystone (Ubuntu) |
|
| 2018-01-10 01:10:38 |
Billy Olsen |
bug task added |
|
cloud-archive |
|
| 2018-01-10 01:10:58 |
Billy Olsen |
nominated for series |
|
cloud-archive/mitaka |
|
| 2018-01-10 01:11:37 |
Billy Olsen |
nominated for series |
|
Ubuntu Xenial |
|
| 2018-01-10 18:24:17 |
Corey Bryant |
bug task added |
|
cloud-archive/mitaka |
|
| 2018-01-10 18:24:25 |
Corey Bryant |
bug task added |
|
keystone (Ubuntu Xenial) |
|
| 2018-01-10 18:26:17 |
Corey Bryant |
cloud-archive/mitaka: status |
New |
Triaged |
|
| 2018-01-10 18:26:19 |
Corey Bryant |
keystone (Ubuntu Xenial): status |
New |
Triaged |
|
| 2018-01-10 18:26:21 |
Corey Bryant |
cloud-archive/mitaka: importance |
Undecided |
High |
|
| 2018-01-10 18:26:25 |
Corey Bryant |
keystone (Ubuntu Xenial): importance |
Undecided |
High |
|
| 2018-01-10 19:01:04 |
Corey Bryant |
nominated for series |
|
cloud-archive/newton |
|
| 2018-01-10 19:01:04 |
Corey Bryant |
bug task added |
|
cloud-archive/newton |
|
| 2018-01-10 19:01:12 |
Corey Bryant |
cloud-archive/newton: status |
New |
Triaged |
|
| 2018-01-10 19:01:15 |
Corey Bryant |
cloud-archive/newton: importance |
Undecided |
High |
|
| 2018-01-10 21:29:34 |
Billy Olsen |
attachment added |
|
newton-lp1582585.debdiff https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1582585/+attachment/5034732/+files/newton-lp1582585.debdiff |
|
| 2018-01-10 21:30:29 |
Billy Olsen |
attachment added |
|
mitaka-lp1582585.debdiff https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1582585/+attachment/5034743/+files/mitaka-lp1582585.debdiff |
|
| 2018-01-11 00:21:18 |
Ubuntu Foundations Team Bug Bot |
tags |
|
patch |
|
| 2018-01-11 00:21:25 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2018-01-11 17:10:07 |
Billy Olsen |
tags |
patch |
patch sts |
|
| 2018-01-11 18:00:55 |
Corey Bryant |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-01-11 18:33:36 |
Corey Bryant |
cloud-archive: status |
New |
Invalid |
|
| 2018-01-11 18:33:40 |
Corey Bryant |
keystone (Ubuntu): status |
New |
Invalid |
|
| 2018-01-12 13:50:38 |
Corey Bryant |
cloud-archive/newton: status |
Triaged |
Fix Committed |
|
| 2018-01-12 13:50:40 |
Corey Bryant |
tags |
patch sts |
patch sts verification-newton-needed |
|
| 2018-01-18 15:16:56 |
Łukasz Zemczak |
description |
In our project, the speed of query user from ldap server is very slow,our ldap user number is 12,000,the query costs almost 45 seconds
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time later,it also goes to db,not use the cache.
So adding the cache to improve the query speed
After adding @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580.
First query time almost costs 50 seconds,but second query time later it only costs 7 seconds.
So it is very necessary to improve this feature |
[Impact]
* When using an LDAP backend for Keystone, the performance can be slow if there are
a large number of users using the cloud. This is due in large part to querying the
SQL database for the identity mapping information of each user in a separate transaction.
For example, an environment with 12,000 users will result in 12,000 sql queries to the
backend database in order to fulfill a user list request. This causes some admin
functions in Horizon UI to take several minutes, which often exceeds the WSGI and any
haproxy timeouts configured.
* This is fixed by backporting a series of patches which caches previously fetched identity
mapping information in a memcached instance and changes the logic to query all of the
user id mapping by the domain the id mapping is in. Additionally, the keystone-manage
command to sync the id mapping information with a backend database in an offline manner
is included to allow offline syncing of the data.
[Test Case]
* Install keystone using an ldap backend w/ large number of users.
* List user information: openstack user list --domain <domain_id>
* observe slow down
[Regression Potential]
* For Mitaka, the caching backends such as memcached or mongodb will likely see more
usage and an increased footprint due to additional data being cached. Caching the
identity mapping information is now standard since Newton and no major issues have
been seen coming from this.
* This code affects the identity mapping between keystone user and the ldap user
(essentially the bridge between the two). While it does not functionally alter the
information that is mapped (e.g. no difference in how the identity mapping is calculated),
it does alter a key code path for information regarding user identity mappings.
[Other Info]
* These patches have been run and tested in a staging environment to production and
have had exposure in the Mitaka path for approximately one month to show their stability.
[Original Description]
In our project, the speed of query user from ldap server is very slow,our ldap user number is 12,000,the query costs almost 45 seconds
The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time later,it also goes to db,not use the cache.
So adding the cache to improve the query speed
After adding @MEMOIZE to the following function
https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580.
First query time almost costs 50 seconds,but second query time later it only costs 7 seconds.
So it is very necessary to improve this feature |
|
| 2018-01-18 15:35:39 |
Łukasz Zemczak |
keystone (Ubuntu Xenial): status |
Triaged |
Fix Committed |
|
| 2018-01-18 15:35:45 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2018-01-18 15:35:50 |
Łukasz Zemczak |
tags |
patch sts verification-newton-needed |
patch sts verification-needed verification-needed-xenial verification-newton-needed |
|
| 2018-01-19 01:58:35 |
Corey Bryant |
cloud-archive/mitaka: status |
Triaged |
Fix Committed |
|
| 2018-01-19 01:58:38 |
Corey Bryant |
tags |
patch sts verification-needed verification-needed-xenial verification-newton-needed |
patch sts verification-mitaka-needed verification-needed verification-needed-xenial verification-newton-needed |
|
| 2018-01-23 00:01:30 |
Brian Murray |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2018-02-09 06:48:17 |
Billy Olsen |
attachment added |
|
keystone-config.tar.gz https://bugs.launchpad.net/keystone/+bug/1582585/+attachment/5051668/+files/keystone-config.tar.gz |
|
| 2018-02-09 06:49:57 |
Billy Olsen |
tags |
patch sts verification-mitaka-needed verification-needed verification-needed-xenial verification-newton-needed |
patch sts verification-done-mitaka verification-newton-done verification-xenial-done |
|
| 2018-02-13 22:45:49 |
Billy Olsen |
tags |
patch sts verification-done-mitaka verification-newton-done verification-xenial-done |
patch sts verification-done-mitaka verification-done-newton verification-done-xenial |
|
| 2018-02-15 07:40:11 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-02-15 07:50:17 |
Launchpad Janitor |
keystone (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2018-02-26 14:49:31 |
Corey Bryant |
cloud-archive/newton: status |
Fix Committed |
Fix Released |
|
| 2018-02-26 14:50:14 |
Corey Bryant |
cloud-archive/mitaka: status |
Fix Committed |
Fix Released |
|
