liberty -> mitaka db migrate fails when constraint names are inconsistent

[Matthew Thode]

specifically 088_domain_specific_roles.py at

    migrate.UniqueConstraint(role_table.c.name,
                             name=_ROLE_NAME_OLD_CONSTRAINT).drop()

If it was easier to translate into actual sql statements I might be able to give more info.

[Morgan Fainberg (mdrnstm)]

This is going to take a bit of work to duplicate - we'll need to get this tested / fixed before mitaka final.

[Matthew Thode (prometheanfire)]

here's what I was getting for 088

ProgrammingError: (psycopg2.ProgrammingError) constraint "ixu_role_name" of relation "role" does not exist
 [SQL: 'ALTER TABLE role DROP CONSTRAINT ixu_role_name']

ProgrammingError: (psycopg2.ProgrammingError) relation "ixu_role_name_domain_id" already exists
 [SQL: 'ALTER TABLE role ADD CONSTRAINT ixu_role_name_domain_id UNIQUE (name, domain_id)']

[Matthew Thode (prometheanfire)]

here is the table after the migration, this had to have manual mangling to get the migration to complete

keystone=> \d role
                                 Table "public.role"
  Column | Type | Modifiers
-----------+------------------------+------------------------------------------------
 id | character varying(64) | not null
 name | character varying(255) | not null
 extra | text |
 domain_id | character varying(64) | not null default '<<null>>'::character varying
Indexes:
    "role_pkey" PRIMARY KEY, btree (id)
    "ixu_role_name_domain_id" UNIQUE CONSTRAINT, btree (name, domain_id)
    "role_name_key" UNIQUE CONSTRAINT, btree (name)
Referenced by:
    TABLE "implied_role" CONSTRAINT "implied_role_implied_role_id_fkey" FOREIGN KEY (implied_role_id) REFERENCES role(id) ON DELETE CASCADE
    TABLE "implied_role" CONSTRAINT "implied_role_prior_role_id_fkey" FOREIGN KEY (prior_role_id) REFERENCES role(id) ON DELETE CASCADE

[Morgan Fainberg (mdrnstm)]

More digging with Matthre Thode's help, turns out the role_name unique constraint is not consistently named. We need to fix the migration to ensure we look for the right constraint and then we need to drop the one that affects only role_name.

This needs to be done as follows:

* Fix migration 88 to drop the right constraint, add a new migration to drop the constraint with a search if it exists

* backport to stable/mitaka

[Morgan Fainberg (mdrnstm)]

This is a release blocker.

[Morgan Fainberg (mdrnstm)]

This isn't just a PGSQL issue it could affect mysql as well.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/298402

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/298406

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300136

[OpenStack Infra (hudson-openstack)]

Change abandoned by Samuel de Medeiros Queiroz (<email address hidden>) on branch: master
Review: https://review.openstack.org/300136
Reason: This isn't necessary anymore. We decided to go with https://review.openstack.org/#/c/298402 instead, as it has been tested in the real deployment where the bug waas observed; this is also RC blocking, and we don't want to take any longer

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/298402
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9e81843719dab3c66d89adf100bb3e1d5c9cedca
Submitter: Jenkins
Branch: master

commit 9e81843719dab3c66d89adf100bb3e1d5c9cedca
Author: Morgan Fainberg <email address hidden>
Date: Mon Mar 28 10:50:12 2016 -0700

    Correct `role_name` constraint dropping

    The `role_name` constraint was not properly dropped in some cases
    because the unique constraint was not consistently named. In all
    cases we must search for the constraint expected, not assume the
    name of the constraint will be consistent (especially from older
    installs that have been moved forward in releases).

    This change fixes migration 88, updates 96 (for backport to
    stable/mitaka), and creates migration 101 for anyone who has moved
    beyond the mitaka -> newton placeholders. This is being overly
    cautious, but specifically is to handle the case where someone
    performed the upgrade and manually fixed the migration resulting
    in duplicated constraints in the role_table.

    Co-Authored-By: "Matthew Thode" <email address hidden>
    Change-Id: Ie0dc3d2449bace57d3e9323b281a2abd2ad0c983
    closes-bug: #1562934

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/298406
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=dba04cdd232ab72704df58cff791d52c1c99bc90
Submitter: Jenkins
Branch: stable/mitaka

commit dba04cdd232ab72704df58cff791d52c1c99bc90
Author: Morgan Fainberg <email address hidden>
Date: Mon Mar 28 10:50:12 2016 -0700

    Correct `role_name` constraint dropping

    The `role_name` constraint was not properly dropped in some cases
    because the unique constraint was not consistently named. In all
    cases we must search for the constraint expected, not assume the
    name of the constraint will be consistent (especially from older
    installs that have been moved forward in releases).

    This change fixes migration 88, updates 96 (for backport to
    stable/mitaka) This is being overly cautious, but specifically
    is to handle the case where someone performed the upgrade and
    manually fixed the migration resulting in duplicated constraints
    in the role_table. Note that migration 100 was not backported since
    it was a placeholder migration.

    Conflicts:
        keystone/tests/unit/test_sql_upgrade.py

    Co-Authored-By: "Matthew Thode" <email address hidden>
    Change-Id: Ie0dc3d2449bace57d3e9323b281a2abd2ad0c983
    closes-bug: #1562934
    (cherry picked from commit 276e57e2083dcad8cbabf9aa9b3bd06c6079c415)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/302567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/302567
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=a78623a456fc0a657a01aa5a887641f0b73d64a8
Submitter: Jenkins
Branch: master

commit 3e5fca06c6b7dd6060721faa39428b133edd10f0
Author: Boris Bobrov <email address hidden>
Date: Tue Apr 5 18:50:48 2016 +0300

    Update federated user display name with shadow_users_api

    When a user comes to the cloud for the first time, a shadow user is
    created. When the user authenticates again, this shadow user is
    fetched and returned. Before it is returned, its display name should
    be updated. But the call to update the display name fails because
    neither identity manager nor identity drivers have the required
    method. However, the required method exists in shadow_users_api.

    The issue was hidden because method shadow_federated_user was
    cached and while the cache lived, the user could authenticate.

    Use the method of shadow_user_api instead of identity_api to update
    federated user display name.

    Change-Id: I58e65bdf3a953f3ded485003939b81f908738e1e
    Closes-Bug: 1566282
    (cherry picked from commit 7ad4f8728cce354617b5facefe5076d65af311c6)

commit dba04cdd232ab72704df58cff791d52c1c99bc90
Author: Morgan Fainberg <email address hidden>
Date: Mon Mar 28 10:50:12 2016 -0700

    Correct `role_name` constraint dropping

    The `role_name` constraint was not properly dropped in some cases
    because the unique constraint was not consistently named. In all
    cases we must search for the constraint expected, not assume the
    name of the constraint will be consistent (especially from older
    installs that have been moved forward in releases).

    This change fixes migration 88, updates 96 (for backport to
    stable/mitaka) This is being overly cautious, but specifically
    is to handle the case where someone performed the upgrade and
    manually fixed the migration resulting in duplicated constraints
    in the role_table. Note that migration 100 was not backported since
    it was a placeholder migration.

    Conflicts:
        keystone/tests/unit/test_sql_upgrade.py

    Co-Authored-By: "Matthew Thode" <email address hidden>
    Change-Id: Ie0dc3d2449bace57d3e9323b281a2abd2ad0c983
    closes-bug: #1562934
    (cherry picked from commit 276e57e2083dcad8cbabf9aa9b3bd06c6079c415)

commit cf1fd9dad7d89d50a10ab204aa487663ea258988
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Mar 30 06:16:27 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Iff9556deb5ebb465c7124c120e7806774c517472

commit c4b08ed2522ee771331d53f153bda5098c9fb9f4
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 29 06:52:52 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I1e81320c932549e38bade5ef8d885ef426990f40

commit dd38543bdc2a11...

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/keystone 10.0.0.0b1 development milestone.