migration 064 causes IndexError: list index out of range

Bug #907878 reported by Dan Prince
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Critical
Dan Prince

Bug Description

Using the latest nova Essex code (starting with commit d503d6b1079f9eafe0430754214fc5b6d4e32c09):

I'm now getting the following exception when trying to run 'nova-manage db sync' with MySQL:

(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-manage", line 2336, in <module>
(nova): TRACE: main()
(nova): TRACE: File "/usr/bin/nova-manage", line 2324, in main
(nova): TRACE: fn(*fn_args, **fn_kwargs)
(nova): TRACE: File "/usr/bin/nova-manage", line 1161, in sync
(nova): TRACE: return migration.db_sync(version)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/migration.py", line 35, in db_sync
(nova): TRACE: return IMPL.db_sync(version=version)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migration.py", line 51, in db_sync
(nova): TRACE: return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
(nova): TRACE: return _migrate(url, repository, version, upgrade=True, err=err, **opts)
(nova): TRACE: File "<string>", line 2, in _migrate
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
(nova): TRACE: return f(*a, **kw)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 365, in _migrate
(nova): TRACE: schema.runchange(ver, change, changeset.step)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 91, in runchange
(nova): TRACE: change.run(self.engine, step)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py", line 145, in run
(nova): TRACE: script_func(engine)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migrate_repo/versions/064_change_instance_id_to_uuid_in_instance_actions.py", line 55, in upgrade
(nova): TRACE: fkey_name = list(instance_actions.c.instance_id.foreign_keys)[0].\
(nova): TRACE: IndexError: list index out of range

Dan Prince (dan-prince)
Changed in nova:
assignee: nobody → Dan Prince (dan-prince)
importance: Undecided → Critical
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/2565

Revision history for this message
Dan Prince (dan-prince) wrote :
Revision history for this message
Lorin Hochstein (lorinh) wrote :

It looks like this is blowing up things in SmokeStack as well, e.g.: http://smokestack.openstack.org/?go=/jobs/6931

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/2565
Committed: http://github.com/openstack/nova/commit/2fcbe7e3ab4925ed627bc0e32676e2b6ff98ff3e
Submitter: Jenkins
Branch: master

commit 2fcbe7e3ab4925ed627bc0e32676e2b6ff98ff3e
Author: Dan Prince <email address hidden>
Date: Thu Dec 22 14:50:36 2011 -0500

    Remove code in migration 064 to drop an fkey that does not exist.
    Fixes LP bug #907878.

    Change-Id: I2e33fe3a58a34c9606b24eafc319afa9fd9a5d86

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
Adam Gandelman (gandelman-a) wrote :

A little late here but this revert breaks migration against MySQL using InnoDB: http://paste.ubuntu.com/803153/plain/

See Bug #904888 to see what the original hacky workaround was added to fix. I'm not sure what the root problem is, but it seems to be a recurring theme in the the DB migrations.

Revision history for this message
Vish Ishaya (vishvananda) wrote :

so looks like we need to introspect table type and only do the drop on innodb?

Revision history for this message
Adam Gandelman (gandelman-a) wrote :

Well, I believe at this point in the migrations, all mysql tables are innodb as they are converted in an earlier migration 022_set_engine_mysql_innodb.py. The issue seems to only trigger when mysql is configured as innodb prior to the initial migrations (which is the default now in Ubuntu as of Precise). The commit that this reverts appears to go about in a better way than the hack before, in that it attempts to manually find the correct FK to drop instead of relying on sqlalchemy to resolve it automatically (which apparently it cannot do in the case where InnoDB was used from the get-go)

Revision history for this message
Thierry Carrez (ttx) wrote :

Opened bug 917205 to track the regression fix -- FixCommitted bugs like this one tend to drop below radar. Please comment there.

Thierry Carrez (ttx)
Changed in nova:
milestone: none → essex-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: essex-3 → 2012.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.