Add a migration for plugins which added L3 support in Grizzly

[Salvatore Orlando]

I am observing the following failure when migrating from folsom to Grizzly:

INFO [alembic.migration] Running upgrade 363468ac592c -> 511471cc46b
INFO [alembic.migration] Running upgrade 511471cc46b -> 3b54bf9e29f7
INFO [alembic.migration] Running upgrade 3b54bf9e29f7 -> 4692d074d587
Traceback (most recent call last):
  File "./bin/quantum-db-manage", line 26, in <module>
    main()
  File "/opt/stack/quantum/quantum/db/migration/cli.py", line 137, in main
    CONF.command.func(config, CONF.command.name)
  File "/opt/stack/quantum/quantum/db/migration/cli.py", line 76, in do_upgrade_downgrade
    do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
  File "/opt/stack/quantum/quantum/db/migration/cli.py", line 55, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
    script.run_env()
  File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 191, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 186, in load_python_file
    module = imp.load_source(module_id, path, open(path, 'rb'))
  File "/opt/stack/quantum/quantum/db/migration/alembic_migrations/env.py", line 100, in <module>
    run_migrations_online()
  File "/opt/stack/quantum/quantum/db/migration/alembic_migrations/env.py", line 84, in run_migrations_online
    options=build_options())
  File "<string>", line 7, in run_migrations
  File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 494, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 211, in run_migrations
    change(**kw)
  File "/opt/stack/quantum/quantum/db/migration/alembic_migrations/versions/4692d074d587_agent_scheduler.py", line 70, in upgrade
    sa.PrimaryKeyConstraint('id')
  File "<string>", line 7, in create_table
  File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 570, in create_table
    self._table(name, *columns, **kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 148, in create_table
    self._exec(schema.CreateTable(table))
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 75, in _exec
    conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.OperationalError: (OperationalError) (1005, "Can't create table 'quantum_nvp.routerl3agentbindings' (errno: 150)") '\nCREATE TABLE routerl3agentbindings (\n\tid VARCHAR(36) NOT NULL, \n\trouter_id VARCHAR(36), \n\tl3_agent_id VARCHAR(36), \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(l3_agent_id) REFERENCES agents (id) ON DELETE CASCADE, \n\tFOREIGN KEY(router_id) REFERENCES routers (id) ON DELETE CASCADE\n)\n\n' ()
user@nvp-os-ctrl:/opt/stack/quantum$ vi quantum/db/migration/alembic_migrations/versions/4692d074d587_agent_scheduler.py

It looks like the problem is related to the fact that the 'routers' table is not in the database as I am running migrations for the NVP plugin which is not L3 capable in folsom.

Setting to grizzly-rc-1 just to keep it on the radar until it's confirmed or rejected.

[Salvatore Orlando (salvatore-orlando)]

the problem seems actually not to be related to this specific migration.
It's unfortunately a bit tricker.

Basically if a plugin was running folsom without extension 'X', and then it's upgraded to Grizzly where X is supported, the data model changes for including support for the 'X' extension are not executed by alembic, and therefore migrations which refer to bits in the data model pertaining to this extension fail.

In my case, this migration failed because folsom NVP plugin did not support L3 extension, and so there was no 'router' table in the database.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24107

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24107
Committed: http://github.com/openstack/quantum/commit/90abfc6b04f1acf61e8a8b3413a90d2a12de68b8
Submitter: Jenkins
Branch: master

commit 90abfc6b04f1acf61e8a8b3413a90d2a12de68b8
Author: Salvatore Orlando <email address hidden>
Date: Mon Mar 11 19:33:01 2013 +0100

    Add l3 db migration for plugins which did not support in folsom

    Bug 1153585

    Several plugins added l3 support in the grizzly release. This means that
    when upgrading from the folsom data model, the relevant data model changes
    should be applied.
    In order to do so this patch refactors the initial migration in order
    to be able to reuse the upgrade_l3 and downgrade_l3 routines.

    Change-Id: I876f6d637665e10fb09d2084bb8a545ce469fbf3