DB migration problem with federation extension

[Marco Fargetta]

Running a keystone-manage db_sync with extension federation fails if the tables already exist.

See http://paste.openstack.org/show/182962/

The problem is that the table created to support the federation are not forced to be in utf8 but the migration script check they are in utf8 and fails otherwise.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/159803

[Boris Bobrov (bbobrov)]

Sorry, but I don't run into this bug. In session http://paste.openstack.org/show/182968/ I pull latest master, downgrade the table and upgrade it, and it works OK.

Maybe there are some special cases where the bug occurs?

[Marco Fargetta (fmarco76)]

In your test you remote the table by downgrade. If you do not downgrade but run again the db_sync you should get the error (existing installation will not remove the DB before to upgrade it).

[Boris Bobrov (bbobrov)]

OK, should I hit the bug if I do "db_sync --extension federation 4" and after that "db_sync --extension federation 5"?

[Marco Fargetta (fmarco76)]

Yes, if I do before 4 and after 5 I get the error

[Boris Bobrov (bbobrov)]

I don't. http://paste.openstack.org/show/183020/

[Marco Fargetta (fmarco76)]

It is very strange. I repeated the test many time and always with the same result:

http://paste.openstack.org/show/183030/

Is the table in you DB created as utf8 or latin (show create table identity_provider)?

[Boris Bobrov (bbobrov)]

http://paste.openstack.org/show/183039/ . That's MySQL in default configuration from Debian stable.

[Marco Fargetta (fmarco76)]

I am analysing the reasons of this difference between my test and Boris test. If I create the keystone DB with utf8 default encoding it works properly otherwise not. Devstack creates the DB with utf8 as default.

In the guideline there is not indication that utf8 is requested. Therefore, we should specify utf8 in all the table as it is except the 3 for the federation.

[Marco Fargetta (fmarco76)]

If I do not specify nothing my database default is latin1, this is the reason of the difference. I am on Ubuntu and there is not specification for the encoding (in my installation). If Debian put utf8 as default then you should not get any error in any test with the DB for this utf8 problem.

[Boris Bobrov (bbobrov)]

OK, I hit the bug now. And my "show create table identity_provider" now shows collation latin1. I have absolutely no idea why it was utf8 in the paste above.

But. I don't understand, why I hit the bug when I db_sync from 0 to 1 and then from 1 to 2 and don't when I do db_sync 2.

[Marco Fargetta (fmarco76)]

The reason is that before any change in the DB there is a conformity chack of all the tables and if it is OK than the downgrade/upgrade is performed. There is not check after. As a result if you create a table with some problems it is verified the next time you perform the upgrade/downgrade. So when you run db_sync <any_value> there are not table and that is OK, if you run again there is the sanity check and it fail.

[Morgan Fainberg (mdrnstm)]

@Borris,

Likely your DB has correct defaults set. This can occur when the defaults for the DB installation is not innodb/utf8

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/159803
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=684a0db51cceba7e62609cc02dd9f18bbef62e5c
Submitter: Jenkins
Branch: master

commit 684a0db51cceba7e62609cc02dd9f18bbef62e5c
Author: Marco Fargetta <email address hidden>
Date: Fri Feb 27 12:43:48 2015 +0100

    Adding utf8 to federation tables

    Tables have to be in utf8 for the migration utility
    to work properly. This fixes the UTF8 issues with the
    federation tables in-line and should be able to be
    backported. A subsequent change to handle sanity checks
    in a more sane way (post migrations) will be implemented
    going forward.

    Change-Id: Id68c3f307f53b6a4de011e2608d899d044aced68
    Closes-Bug: 1426334