Duplicated entries in users API

Bug #1848342 reported by Pedro Henrique Pereira Martins on 2019-10-16
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Medium
Pedro Henrique Pereira Martins

Bug Description

System version
==============
Keystone: 14.0.1 (Rocky)

The error happens when I try to show a federated user via the user name as the key instead of using the ID

Reproducing the error:

======================

First of all, I created a user in my Identity Provider (Idp) that is configured my KeyStone instance.

We use OpenID Connect protocol to create the federated environment. In this test, the user created is

 named as 'use_case'. At this moment, the user is not in the OpenStack database, just in the Idp

 (ephemeral user type).

Authenticating with the ephemeral federated user:

    Let's see the database tables before the login process.

    Users already created in the Idp (before the login):

        select * from user;

        +----------------------------------+-------+---------+--------------------+---------------------+----------------+-----------+

        | id | extra | enabled | default_project_id | created_at | last_active_at | domain_id |

        +----------------------------------+-------+---------+--------------------+---------------------+----------------+-----------+

        | 1b0ed400ec974420a3574a8788acc795 | {} | 1 | NULL | 2019-09-16 20:06:39 | NULL | default |

        +----------------------------------+-------+---------+--------------------+---------------------+----------------+-----------+

        select * from local_user;

        +-----+----------------------------------+-----------+-------+-------------------+----------------+

        | id | user_id | domain_id | name | failed_auth_count | failed_auth_at |

        +-----+----------------------------------+-----------+-------+-------------------+----------------+

        | 1 | 1b0ed400ec974420a3574a8788acc795 | default | admin | 0 | NULL |

        +-----+----------------------------------+-----------+-------+-------------------+----------------+

        select * from federated_user;

        Empty set (0.00 sec)

    So far, we have only the local user admin

Then, we execute the login using the ephemeral user that we created in the IdP.

    Let's the database state after the first login:

        select * from user;

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        | id | extra | enabled | default_project_id | created_at | last_active_at | domain_id |

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        | 1b0ed400ec974420a3574a8788acc795 | {} | 1 | NULL | 2019-09-16 20:06:39 | NULL | default |

        | f0327d98278f4b1881aec9c051b027d3 | {"email": "<email address hidden>"} | 1 | NULL | 2019-10-15 18:08:17 | NULL | d28c2423e9b546deb37a034bb2134f4d |

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        select * from local_user;

        +----+----------------------------------+-----------+-------+-------------------+----------------+

        | id | user_id | domain_id | name | failed_auth_count | failed_auth_at |

        +----+----------------------------------+-----------+-------+-------------------+----------------+

        | 1 | 1b0ed400ec974420a3574a8788acc795 | default | admin | 0 | NULL |

        +----+----------------------------------+-----------+-------+-------------------+----------------+

        select * from federated_user;

        +----+----------------------------------+--------+-------------+-----------+--------------+

        | id | user_id | idp_id | protocol_id | unique_id | display_name |

        +----+----------------------------------+--------+-------------+-----------+--------------+

        | 1 | f0327d98278f4b1881aec9c051b027d3 | my_idp | openid | use_case | use_case |

        +----+----------------------------------+--------+-------------+-----------+--------------+

        Here we can see that the user did not exist in OpenStack database before the first login; then, KeyStone receives the user attributes (during the authentication process) from the Idp and register the user in the OpenStack database as a federated user.

        (user and federated_user tables).

        Here is the OpenStack client output (when executing the show command after the first login):

        openstack user show use_case

        +---------------------+----------------------------------+

        | Field | Value |

        +---------------------+----------------------------------+

        | domain_id | d28c2423e9b546deb37a034bb2134f4d |

        | email | <email address hidden> |

        | enabled | True |

        | id | f0327d98278f4b1881aec9c051b027d3 |

        | name | use_case |

        | options | {} |

        | password_expires_at | None |

        +---------------------+----------------------------------+

    Afterwards, we logout and then we login again in Horizon via the IdP. Please invalidate your session, OpenStack has a bug, and it is not invalidating IdP sessions when the logout is called in Horizon):

        select * from user;

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        | id | extra | enabled | default_project_id | created_at | last_active_at | domain_id |

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        | 1b0ed400ec974420a3574a8788acc795 | {} | 1 | NULL | 2019-09-16 20:06:39 | NULL | default |

        | f0327d98278f4b1881aec9c051b027d3 | {"email": "<email address hidden>"} | 1 | NULL | 2019-10-15 18:08:17 | NULL | d28c2423e9b546deb37a034bb2134f4d |

        +----------------------------------+---------------------------------+---------+--------------------+---------------------+----------------+----------------------------------+

        select * from local_user;

        +----+----------------------------------+----------------------------------+----------+-------------------+----------------+

        | id | user_id | domain_id | name | failed_auth_count | failed_auth_at |

        +----+----------------------------------+----------------------------------+----------+-------------------+----------------+

        | 1 | 1b0ed400ec974420a3574a8788acc795 | default | admin | 0 | NULL |

        | 4 | f0327d98278f4b1881aec9c051b027d3 | d28c2423e9b546deb37a034bb2134f4d | use_case | 0 | NULL |

        +----+----------------------------------+----------------------------------+----------+-------------------+----------------+

        select * from federated_user;

        +----+----------------------------------+--------+-------------+-----------+--------------+

        | id | user_id | idp_id | protocol_id | unique_id | display_name |

        +----+----------------------------------+--------+-------------+-----------+--------------+

        | 1 | f0327d98278f4b1881aec9c051b027d3 | my_idp | openid | use_case | use_case |

        +----+----------------------------------+--------+-------------+-----------+--------------+

        Here is the bug, the user already existed in the OpenStack database, and during the second login, Keystone tried to update data with the attributes that come from the IdP. During the update process, Keystone will create an entry in the local_user table.

        The code that is generating this behavior can be found in https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L1421

        After that, if I try to show the user via OpenStack client, I receive the following error

        openstack user show use_case

        More than one user exists with the name 'use_case'.

Here is an excerpt of our Attribute Mapping JSON:

    [

    {

        "local": [

        {

            "user": {

            "name": "{0}",

            "email": "{1}",

            "domain": {

                "name": "{2}"

            }

            },

            "domain": {

            "name": "{2}"

            },

            "projects": [

            {

                "name": "{3}",

                "roles": [

                {

                    "name": "member"

                },

                {

                    "name": "domadmin"

                },

                {

                    "name": "heat_stack_owner"

                },

                {

                    "name": "load-balancer_member"

                }

                ]

            }

            ]

        }

        ],

        "remote": [

        {

            "type": "OIDC-preferred_username"

        },

        {

            "type": "OIDC-email"

        },

        {

            "type": "OIDC-openstack-user-domain"

        },

        {

            "type": "OIDC-openstack-default-project"

        },

        {

            "type": "OIDC-openstack-user-role",

            "any_one_of": [

            "domain-admin"

            ]

        }

        ]

    },

    {

        "local": [

        {

            "user": {

            "name": "{0}",

            "email": "{1}",

            "type": "local",

            "domain": {

                "name": "{2}"

            }

            },

            "domain": {

            "name": "{2}"

            },

            "projects": [

            {

                "name": "{3}",

                "roles": [

                {

                    "name": "member"

                },

                {

                    "name": "heat_stack_owner"

                },

                {

                    "name": "load-balancer_member"

                }

                ]

            }

            ]

        }

        ],

        "remote": [

        {

            "type": "OIDC-preferred_username"

        },

        {

            "type": "OIDC-email"

        },

        {

            "type": "OIDC-openstack-user-domain"

        },

        {

            "type": "OIDC-openstack-default-project"

        },

        {

            "type": "OIDC-openstack-user-status",

            "any_one_of": [

            "local"

            ]

        },

        {

            "type": "OIDC-openstack-user-role",

            "any_one_of": [

            "member"

            ]

        }

        ]

    }

    ]

Here is a link to the PR that propose to fix that issue:

https://review.opendev.org/#/c/687990

Changed in keystone:
assignee: nobody → Pedro Henrique Pereira Martins (pedrohpmartins)
status: New → In Progress
Colleen Murphy (krinkle) wrote :

Thanks for the detailed bug report, I've managed to reproduce the issue.

The problem is not that the entries are duplicated in list output, that is just a symptom. The problem is that an ephemeral federated user is ending up being created in the local_user table, which should never happen. In this case, it's because your mapping has the "email" property in the user, and it's hitting this case:

https://opendev.org/openstack/keystone/src/commit/f9a086e16599123fba8a12c1c06bec73565a0ebc/keystone/identity/core.py#L1419-L1422

It's running the update_user method on the identity sql driver. Even though the local user doesn't exist, since it's running against the user table, it successfully finds the shadow user:

https://opendev.org/openstack/keystone/src/commit/f9a086e16599123fba8a12c1c06bec73565a0ebc/keystone/identity/backends/sql.py#L206

but then for some reason recreates the record in the local_user table:

https://opendev.org/openstack/keystone/src/commit/f9a086e16599123fba8a12c1c06bec73565a0ebc/keystone/identity/backends/sql.py#L210

So we'll need to fix that, rather than filter the output.

(For anyone else trying to reproduce this, it helps to disable keystone caching, otherwise the issue only appears intermittently.)

Changed in keystone:
status: In Progress → Triaged
importance: Undecided → Medium
Changed in keystone:
status: Triaged → In Progress
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers