MySQL db_sync issue

Bug #1098174 reported by Jacob Godin
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
Medium
Dolph Mathews

Bug Description

Keystone will not complete its install with MySQL db_sync. Bug produced on a new install.

The first db_sync returns:
# keystone-manage db_sync
Traceback (most recent call last):
  File "/usr/bin/keystone-manage", line 5, in <module>
    pkg_resources.run_script('keystone==2013.1', 'keystone-manage')
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
    execfile(script_filename, namespace, namespace)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/EGG-INFO/scripts/keystone-manage", line 28, in <module>
    cli.main(argv=sys.argv, config_files=config_files)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 144, in main
    CONF.command.cmd_class.main()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 49, in main
    driver.db_sync()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/identity/backends/sql.py", line 167, in db_sync
    migration.db_sync()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migration.py", line 52, in db_sync
    return versioning_api.upgrade(CONF.sql.connection, repo_path, version)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
    return f(*a, **kw)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 366, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 91, in runchange
    change.run(self.engine, step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migrate_repo/versions/010_endpoints_v3.py", line 41, in upgrade
    new_table.create(migrate_engine, checkfirst=True)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 593, in create
    checkfirst=checkfirst)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 2302, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1972, in _run_visitor
    **kwargs).traverse_single(element)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 106, in traverse_single
    return meth(obj, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/ddl.py", line 86, in visit_table
    self.connection.execute(schema.CreateTable(table))
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _execute_ddl
    compiled = ddl.compile(dialect=dialect)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/expression.py", line 1778, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 2908, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 705, in __init__
    self.string = self.process(self.statement)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 724, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 72, in _compiler_dispatch
    return getter(visitor)(self, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1612, in visit_create_table
    const = self.create_table_constraints(table)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 1410, in create_table_constraints
    constraint_string = super(MySQLDDLCompiler, self).create_table_constraints(table)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1632, in create_table_constraints
    for constraint in constraints
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1630, in <genexpr>
    return ", \n\t".join(p for p in
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1638, in <genexpr>
    not getattr(constraint, 'use_alter', False)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 724, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 72, in _compiler_dispatch
    return getter(visitor)(self, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1777, in visit_foreign_key_constraint
    remote_table = list(constraint._elements.values())[0].column.table
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 494, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 1392, in column
    tname)
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'endpoint_v3.service_id' could not find table 'service' with which to generate a foreign key to target column 'id'

The second (and subsequent) runs return:
# keystone-manage db_sync
Traceback (most recent call last):
  File "/usr/bin/keystone-manage", line 5, in <module>
    pkg_resources.run_script('keystone==2013.1', 'keystone-manage')
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
    execfile(script_filename, namespace, namespace)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/EGG-INFO/scripts/keystone-manage", line 28, in <module>
    cli.main(argv=sys.argv, config_files=config_files)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 144, in main
    CONF.command.cmd_class.main()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 49, in main
    driver.db_sync()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/identity/backends/sql.py", line 167, in db_sync
    migration.db_sync()
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migration.py", line 52, in db_sync
    return versioning_api.upgrade(CONF.sql.connection, repo_path, version)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
    return f(*a, **kw)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 366, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 91, in runchange
    change.run(self.engine, step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migrate_repo/versions/010_endpoints_v3.py", line 25, in upgrade
    legacy_table = sql.Table('endpoint', meta, autoload=True)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 318, in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 381, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 408, in _autoload
    self, include_columns, exclude_columns
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 2425, in run_callable
    return conn.run_callable(callable_, *args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1968, in run_callable
    return callable_(self, *args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 260, in reflecttable
    return insp.reflecttable(table, include_columns, exclude_columns)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 350, in reflecttable
    tbl_opts = self.get_table_options(table_name, schema, **table.kwargs)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 178, in get_table_options
    **kw)
  File "<string>", line 1, in <lambda>
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 47, in cache
    ret = fn(self, con, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2064, in get_table_options
    parsed_state = self._parsed_state_or_create(connection, table_name, schema, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2163, in _parsed_state_or_create
    info_cache=kw.get('info_cache', None)
  File "<string>", line 1, in <lambda>
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 47, in cache
    ret = fn(self, con, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2188, in _setup_parser
    full_name=full_name)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2275, in _show_create_table
    raise exc.NoSuchTableError(full_name)
sqlalchemy.exc.NoSuchTableError: `endpoint`

The DB ends up looking like:

+------------------------+
| Tables_in_keystone |
+------------------------+
| credential |
| domain |
| ec2_credential |
| endpoint_v2 |
| metadata |
| migrate_version |
| policy |
| role |
| service |
| tenant |
| token |
| user |
| user_domain_metadata |
| user_tenant_membership |
+------------------------+

Dolph Mathews (dolph)
Changed in keystone:
assignee: nobody → Dolph Mathews (dolph)
Revision history for this message
Jacob Godin (jacobgodin) wrote :

Some info:

# uname -a
Linux keystone_1 2.6.32-279.19.1.el6.x86_64 #1 SMP Wed Dec 19 07:05:20 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release
CentOS release 6.3 (Final)

MySQL - Server version: 5.1.66 Source distribution

# python -c "import sqlalchemy; print sqlalchemy.__version__"
0.7.9

# python -c "import migrate; print migrate.__version__"
0.7.2

Dolph Mathews (dolph)
Changed in keystone:
status: New → Confirmed
Revision history for this message
Dolph Mathews (dolph) wrote :

The failure leaves your migrate_version at 9, as migration 10 fails. To undo the partially completed migration, run:

  RENAME TABLE endpoint_v2 TO endpoint;

Changed in keystone:
status: Confirmed → In Progress
Dolph Mathews (dolph)
Changed in keystone:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/19667
Committed: http://github.com/openstack/keystone/commit/57f1e3011c05b5a853b506bb1dae4fbf75df517a
Submitter: Jenkins
Branch: master

commit 57f1e3011c05b5a853b506bb1dae4fbf75df517a
Author: Dolph Mathews <email address hidden>
Date: Mon Jan 14 15:40:28 2013 -0600

    Autoload schema before creating FK's (bug 1098174)

    The 'service' schema must be loaded before sqlalchemy-migrate can create
    ForeignKeys referencing it.

    Change-Id: Icd87d40f4c01e4daec0fdb7a6ad93847958236ec

Changed in keystone:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in keystone:
milestone: none → grizzly-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: grizzly-3 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.