Grizzly-havana upgrade fails when migrating from stamped db

[Jakub Libosvar]

If grizzly is deployed without using quantum-db-manage and letting neutron to create tables, there are not created tables servicedefinitions and servicetypes. These tables are dropped later when using LoadBalancerPlugin. When creating db scheme with quantum-db-manage, these tables are created and dropped correctly.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/69433

[Jakub Libosvar (libosvar)]

There is also another failure:
No handlers could be found for logger "neutron.common.legacy"
INFO [alembic.migration] Context impl MySQLImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
No handlers could be found for logger "neutron.common.legacy"
INFO [alembic.migration] Context impl MySQLImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
INFO [alembic.migration] Running upgrade grizzly -> f489cf14a79c
Traceback (most recent call last):
  File "/usr/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/cli.py", line 143, in main
    CONF.command.func(config, CONF.command.name)
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/cli.py", line 80, in do_upgrade_downgrade
    do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/cli.py", line 59, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/alembic/command.py", line 124, in upgrade
    script.run_env()
  File "/usr/lib/python2.6/site-packages/alembic/script.py", line 191, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/lib/python2.6/site-packages/alembic/util.py", line 186, in load_python_file
    module = imp.load_source(module_id, path, open(path, 'rb'))
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/alembic_migrations/env.py", line 105, in <module>
    run_migrations_online()
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/alembic_migrations/env.py", line 89, in run_migrations_online
    options=build_options())
  File "<string>", line 7, in run_migrations
  File "/usr/lib/python2.6/site-packages/alembic/environment.py", line 494, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/python2.6/site-packages/alembic/migration.py", line 211, in run_migrations
    change(**kw)
  File "/usr/lib/python2.6/site-packages/neutron/db/migration/alembic_migrations/versions/f489cf14a79c_lbaas_havana.py", line 61, in upgrade
    sa.PrimaryKeyConstraint(u'id')
  File "<string>", line 7, in create_table
  File "/usr/lib/python2.6/site-packages/alembic/operations.py", line 570, in create_table
    self._table(name, *columns, **kw)
  File "/usr/lib/python2.6/site-packages/alembic/ddl/impl.py", line 148, in create_table
    self._exec(schema.CreateTable(table))
  File "/usr/lib/python2.6/site-packages/alembic/ddl/impl.py", line 75, in _exec
    conn.execute(construct, *multiparams, **params)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
    compiled
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context
    context)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1691, in _execute_conte...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/75924

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/69433
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d26bd87919990923b6fc85779f03705514d8d0b3
Submitter: Jenkins
Branch: master

commit d26bd87919990923b6fc85779f03705514d8d0b3
Author: Jakub Libosvar <email address hidden>
Date: Mon Jan 27 19:20:51 2014 +0100

    Drop service* tables only if they exist

    Tables servicedefinitions and servicetypes are not used by neutron but
    created using db migration tool. If quantum-server in grizzly was
    started without using migration tool then tables mentioned above are not
    created. During migration from grizzly to havana alembic attempts to
    drop these unused tables. This patch uses "IF EXISTS" clause when
    dropping mentioned tables.

    Closes-bug: #1273355
    Change-Id: I33fcb3e1dc96ce37ec9c00987cb5a3a953ca691d

[Jakub Libosvar (libosvar)]

The error from comment #2 is still not fixed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/128239

[Thierry Carrez (ttx)]

@Jakub: Is https://review.openstack.org/128239 what needs to be done to fix comment #2 ? If yes, i adjusted tasks to track the icehouse backport. If not, for sake of tracking, you should file a separate bug to track what's left to do on Juno/Kilo

[Jakub Libosvar (libosvar)]

Yes, https://review.openstack.org/128239 is fixing comment #2. AFAIK we don't support upgrade from Grizzly to Juno so there is no action needed for Juno/Kilo.

[Thierry Carrez (ttx)]

Awesome, thanks. It should get picked up in the next icehouse release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/128239
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=965fb179d47cd8355aafe5ee6a61f434b84f1639
Submitter: Jenkins
Branch: stable/icehouse

commit 965fb179d47cd8355aafe5ee6a61f434b84f1639
Author: Jakub Libosvar <email address hidden>
Date: Mon Oct 13 17:21:36 2014 +0200

    Skip lbaas table creation if tables already exist

    This patch makes possible upgrade from Grizzly to Icehouse
    in case LoadBalancer service plugin was used in Grizzly. LoadBalancer
    tables were introduced in Grizzly therefore they necessarily don't
    need to be created for Havana and beyond.

    This patch goes to Icehouse stable branch only because of all migration
    scripts till Havana were squashed together in Juno release. That means
    modified file in this patch no longer exists in current master branch.

    Change-Id: I09ed3e4b142ccde44cf2e92184d0d92d898a0284
    Closes-bug: 1273355