Upgrades from Austin fail

[Soren Hansen]

Attempting to go from Austin to Bexar, yields:

root@umltest:~# nova-manage db sync
2011-01-28 11:29:36,011 INFO migrate.versioning.api [-] 0 -> 1...
2011-01-28 11:29:36,042 INFO nova.root [-] Table('auth_tokens', MetaData(Engine(sqlite:////var/lib/nova/nova.sqlite)), Column('created_at', DateTime(timezone=False), table=<auth_tokens>), Column('updated_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted', Boolean(create_constraint=True, name=None), table=<auth_tokens>), Column('token_hash', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>, primary_key=True, nullable=False), Column('user_id', Integer(), table=<auth_tokens>), Column('server_manageent_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('storage_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('cdn_management_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), schema=None)
2011-01-28 11:29:36,043 ERROR nova.root [-] Exception while creating table
(nova.root): TRACE: Traceback (most recent call last):
(nova.root): TRACE: File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/migrate_repo/versions/001_austin.py", line 532, in upgrade
(nova.root): TRACE: table.create()
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/schema.py", line 439, in create
(nova.root): TRACE: bind.create(self, checkfirst=checkfirst)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1647, in create
(nova.root): TRACE: connection=connection, **kwargs)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1682, in _run_visitor
(nova.root): TRACE: **kwargs).traverse_single(element)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/sql/visitors.py", line 77, in traverse_single
(nova.root): TRACE: return meth(obj, **kw)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/ddl.py", line 58, in visit_table
(nova.root): TRACE: self.connection.execute(schema.CreateTable(table))
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1157, in execute
(nova.root): TRACE: params)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1210, in _execute_ddl
(nova.root): TRACE: return self.__execute_context(context)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1268, in __execute_context
(nova.root): TRACE: context.parameters[0], context=context)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1367, in _cursor_execute
(nova.root): TRACE: context)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1360, in _cursor_execute
(nova.root): TRACE: context)
(nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/default.py", line 288, in do_execute
(nova.root): TRACE: cursor.execute(statement, parameters)
(nova.root): TRACE: OperationalError: (OperationalError) table auth_tokens already exists u'\nCREATE TABLE auth_tokens (\n\tcreated_at DATETIME, \n\tupdated_at DATETIME, \n\tdeleted_at DATETIME, \n\tdeleted BOOLEAN, \n\ttoken_hash VARCHAR(255) NOT NULL, \n\tuser_id INTEGER, \n\tserver_manageent_url VARCHAR(255), \n\tstorage_url VARCHAR(255), \n\tcdn_management_url VARCHAR(255), \n\tPRIMARY KEY (token_hash), \n\tCHECK (deleted IN (0, 1))\n)\n\n' ()
(nova.root): TRACE:
Traceback (most recent call last):
  File "/usr/bin/nova-manage", line 694, in <module>
    main()
  File "/usr/bin/nova-manage", line 686, in main
    fn(*argv)
  File "/usr/bin/nova-manage", line 565, in sync
    return migration.db_sync(version)
  File "/usr/lib/pymodules/python2.6/nova/db/migration.py", line 33, in db_sync
    return IMPL.db_sync(version=version)
  File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/migration.py", line 33, in db_sync
    return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)
  File "/usr/lib/pymodules/python2.6/migrate/versioning/api.py", line 185, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/pymodules/python2.6/migrate/versioning/util/__init__.py", line 160, in with_engine
    return f(*a, **kw)
  File "/usr/lib/pymodules/python2.6/migrate/versioning/api.py", line 364, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/pymodules/python2.6/migrate/versioning/schema.py", line 83, in runchange
    change.run(self.engine, step)
  File "/usr/lib/pymodules/python2.6/migrate/versioning/script/py.py", line 140, in run
    script_func(engine)
  File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/migrate_repo/versions/001_austin.py", line 532, in upgrade
    table.create()
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/schema.py", line 439, in create
    bind.create(self, checkfirst=checkfirst)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1647, in create
    connection=connection, **kwargs)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1682, in _run_visitor
    **kwargs).traverse_single(element)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/sql/visitors.py", line 77, in traverse_single
    return meth(obj, **kw)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/ddl.py", line 58, in visit_table
    self.connection.execute(schema.CreateTable(table))
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1157, in execute
    params)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1210, in _execute_ddl
    return self.__execute_context(context)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1268, in __execute_context
    context.parameters[0], context=context)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1367, in _cursor_execute
    context)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1360, in _cursor_execute
    context)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/default.py", line 288, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (OperationalError) table auth_tokens already exists u'\nCREATE TABLE auth_tokens (\n\tcreated_at DATETIME, \n\tupdated_at DATETIME, \n\tdeleted_at DATETIME, \n\tdeleted BOOLEAN, \n\ttoken_hash VARCHAR(255) NOT NULL, \n\tuser_id INTEGER, \n\tserver_manageent_url VARCHAR(255), \n\tstorage_url VARCHAR(255), \n\tcdn_management_url VARCHAR(255), \n\tPRIMARY KEY (token_hash), \n\tCHECK (deleted IN (0, 1))\n)\n\n' ()

I'm guessing this is because migrate can't tell that it's already at schema version 1.

[Jay Pipes (jaypipes)]

This really should be Critical, not High, IMHO.

Cheers,
jay

On Fri, Jan 28, 2011 at 7:11 AM, Thierry Carrez <email address hidden> wrote:
> ** Changed in: nova
>   Importance: Undecided => High
>
> ** Changed in: nova
>       Status: New => Confirmed
>
> ** Changed in: nova
>    Milestone: None => bexar-rc
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/709183
>
> Title:
>  Upgrades from Austin fail
>

[Thierry Carrez (ttx)]

It's High because you can workaround it... But I agree it's definitely RC material (because it's a painful workaround, and needs to be fixed before release rather than after)

[Devin Carlen (devcamcar)]

This is going to be a common problem for anyone using austin. We should create an upgrade script in tools/.

The problem is the schema exists but there is no obvious way for nova to know this. There is a way in sqlalchemy to "fake" upgrade and have it only update the schema version.

For people running random versions from trunk along the way, this isn't really an option. But we really should go out of our way to provide an official upgrade path using the method describe above, or similar.

Devin

On Jan 28, 2011, at 11:38 AM, Thierry Carrez wrote:

> It's High because you can workaround it... But I agree it's definitely
> RC material (because it's a painful workaround, and needs to be fixed
> before release rather than after)
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/709183
>
> Title:
> Upgrades from Austin fail
>
> Status in OpenStack Compute (Nova):
> Confirmed
>
> Bug description:
> Attempting to go from Austin to Bexar, yields:
>
> root@umltest:~# nova-manage db sync
> 2011-01-28 11:29:36,011 INFO migrate.versioning.api [-] 0 -> 1...
> 2011-01-28 11:29:36,042 INFO nova.root [-] Table('auth_tokens', MetaData(Engine(sqlite:////var/lib/nova/nova.sqlite)), Column('created_at', DateTime(timezone=False), table=<auth_tokens>), Column('updated_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted', Boolean(create_constraint=True, name=None), table=<auth_tokens>), Column('token_hash', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>, primary_key=True, nullable=False), Column('user_id', Integer(), table=<auth_tokens>), Column('server_manageent_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('storage_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('cdn_management_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), schema=None)
> 2011-01-28 11:29:36,043 ERROR nova.root [-] Exception while creating table
> (nova.root): TRACE: Traceback (most recent call last):
> (nova.root): TRACE: File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/migrate_repo/versions/001_austin.py", line 532, in upgrade
> (nova.root): TRACE: table.create()
> (nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/schema.py", line 439, in create
> (nova.root): TRACE: bind.create(self, checkfirst=checkfirst)
> (nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1647, in create
> (nova.root): TRACE: connection=connection, **kwargs)
> (nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1682, in _run_visitor
> (nova.root): TRACE: **kwargs).traverse_single(element)
> (...

[Vish Ishaya (vishvananda)]

There is already a migration script in. This is just a bug in the migration script.

On Jan 28, 2011, at 11:57 AM, Devin Carlen wrote:

> This is going to be a common problem for anyone using austin. We should
> create an upgrade script in tools/.
>
> The problem is the schema exists but there is no obvious way for nova to
> know this. There is a way in sqlalchemy to "fake" upgrade and have it
> only update the schema version.
>
> For people running random versions from trunk along the way, this isn't
> really an option. But we really should go out of our way to provide an
> official upgrade path using the method describe above, or similar.
>
> Devin
>
>
> On Jan 28, 2011, at 11:38 AM, Thierry Carrez wrote:
>
>> It's High because you can workaround it... But I agree it's definitely
>> RC material (because it's a painful workaround, and needs to be fixed
>> before release rather than after)
>>
>> --
>> You received this bug notification because you are a member of Nova Bug
>> Team, which is subscribed to OpenStack Compute (nova).
>> https://bugs.launchpad.net/bugs/709183
>>
>> Title:
>> Upgrades from Austin fail
>>
>> Status in OpenStack Compute (Nova):
>> Confirmed
>>
>> Bug description:
>> Attempting to go from Austin to Bexar, yields:
>>
>> root@umltest:~# nova-manage db sync
>> 2011-01-28 11:29:36,011 INFO migrate.versioning.api [-] 0 -> 1...
>> 2011-01-28 11:29:36,042 INFO nova.root [-] Table('auth_tokens', MetaData(Engine(sqlite:////var/lib/nova/nova.sqlite)), Column('created_at', DateTime(timezone=False), table=<auth_tokens>), Column('updated_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted_at', DateTime(timezone=False), table=<auth_tokens>), Column('deleted', Boolean(create_constraint=True, name=None), table=<auth_tokens>), Column('token_hash', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>, primary_key=True, nullable=False), Column('user_id', Integer(), table=<auth_tokens>), Column('server_manageent_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('storage_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), Column('cdn_management_url', String(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), table=<auth_tokens>), schema=None)
>> 2011-01-28 11:29:36,043 ERROR nova.root [-] Exception while creating table
>> (nova.root): TRACE: Traceback (most recent call last):
>> (nova.root): TRACE: File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/migrate_repo/versions/001_austin.py", line 532, in upgrade
>> (nova.root): TRACE: table.create()
>> (nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/schema.py", line 439, in create
>> (nova.root): TRACE: bind.create(self, checkfirst=checkfirst)
>> (nova.root): TRACE: File "/usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1647, in create
>> (nova.root): TRACE: connection=connection, **kwargs)
>> (...

[termie (termie)]

Fix proposed: https://code.launchpad.net/~termie/nova/austin_migration/+merge/47952