Erroneous success reported (keyerror vernum)

[Michael Still]

https://ssl.rcbops.com/turbo_hipster/logviewer/?q=/turbo_hipster/results/56/56729/1/check/gate-real-db-upgrade_nova_percona_devstack_150/55ff034/datasets_devstack_150.log

was reported as a success, but contains:

2013-12-24 06:10:26,833 [output] + nova-manage --config-file .../jobs/56/56729/1/check/gate-real-db-upgrade_nova_percona_devstack_150/55ff034/nova-trunk.conf --verbose db sync
2013-12-24 06:10:28,818 [output]
2013-12-24 06:10:28,818 [output] Traceback (most recent call last):
2013-12-24 06:10:28,818 [output] File ".../55ff03412ca54a02ac632d24180945e0/bin/nova-manage", line 10, in
2013-12-24 06:10:28,818 [output] sys.exit(main())
2013-12-24 06:10:28,818 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/cmd/manage.py", line 1378, in main
2013-12-24 06:10:28,818 [output] ret = fn(*fn_args, **fn_kwargs)
2013-12-24 06:10:28,819 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/cmd/manage.py", line 886, in sync
2013-12-24 06:10:28,819 [output] return migration.db_sync(version)
2013-12-24 06:10:28,819 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/db/migration.py", line 31, in db_sync
2013-12-24 06:10:28,819 [output] return IMPL.db_sync(version=version)
2013-12-24 06:10:28,819 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/db/sqlalchemy/migration.py", line 46, in db_sync
2013-12-24 06:10:28,819 [output] return versioning_api.upgrade(get_engine(), repository, version)
2013-12-24 06:10:28,819 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
2013-12-24 06:10:28,819 [output] return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2013-12-24 06:10:28,820 [output] File "", line 2, in _migrate
2013-12-24 06:10:28,820 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
2013-12-24 06:10:28,820 [output] return f(*a, **kw)
2013-12-24 06:10:28,820 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 345, in _migrate
2013-12-24 06:10:28,820 [output] changeset = schema.changeset(version)
2013-12-24 06:10:28,820 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/schema.py", line 80, in changeset
2013-12-24 06:10:28,820 [output] changeset = self.repository.changeset(database, start_ver, version)
2013-12-24 06:10:28,820 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/repository.py", line 225, in changeset
2013-12-24 06:10:28,820 [output] changes = [self.version(v).script(database, op) for v in versions]
2013-12-24 06:10:28,820 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/repository.py", line 189, in version
2013-12-24 06:10:28,821 [output] return self.versions.version(*p, **k)
2013-12-24 06:10:28,821 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/version.py", line 155, in version
2013-12-24 06:10:28,821 [output] return self.versions[VerNum(vernum)]
2013-12-24 06:10:28,821 [output] KeyError:
2013-12-24 06:10:28,821 [output]
2013-12-24 06:10:28,821 [output] Traceback (most recent call last):
2013-12-24 06:10:28,821 [output] File ".../55ff03412ca54a02ac632d24180945e0/bin/nova-manage", line 10, in
2013-12-24 06:10:28,821 [output] sys.exit(main())
2013-12-24 06:10:28,821 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/cmd/manage.py", line 1378, in main
2013-12-24 06:10:28,822 [output] ret = fn(*fn_args, **fn_kwargs)
2013-12-24 06:10:28,822 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/cmd/manage.py", line 886, in sync
2013-12-24 06:10:28,822 [output] return migration.db_sync(version)
2013-12-24 06:10:28,822 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/db/migration.py", line 31, in db_sync
2013-12-24 06:10:28,822 [output] return IMPL.db_sync(version=version)
2013-12-24 06:10:28,822 [output] File ".../git/gate_real_db_upgrade-thw11-percona.rcbops.com-1/openstack/nova/nova/db/sqlalchemy/migration.py", line 46, in db_sync
2013-12-24 06:10:28,822 [output] return versioning_api.upgrade(get_engine(), repository, version)
2013-12-24 06:10:28,822 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
2013-12-24 06:10:28,822 [output] return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2013-12-24 06:10:28,823 [output] File "", line 2, in _migrate
2013-12-24 06:10:28,823 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
2013-12-24 06:10:28,823 [output] return f(*a, **kw)
2013-12-24 06:10:28,823 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 345, in _migrate
2013-12-24 06:10:28,823 [output] changeset = schema.changeset(version)
2013-12-24 06:10:28,823 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/schema.py", line 80, in changeset
2013-12-24 06:10:28,823 [output] changeset = self.repository.changeset(database, start_ver, version)
2013-12-24 06:10:28,823 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/repository.py", line 225, in changeset
2013-12-24 06:10:28,823 [output] changes = [self.version(v).script(database, op) for v in versions]
2013-12-24 06:10:28,824 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/repository.py", line 189, in version
2013-12-24 06:10:28,824 [output] return self.versions.version(*p, **k)
2013-12-24 06:10:28,824 [output] File ".../55ff03412ca54a02ac632d24180945e0/local/lib/python2.7/site-packages/migrate/versioning/version.py", line 155, in version
2013-12-24 06:10:28,824 [output] return self.versions[VerNum(vernum)]
2013-12-24 06:10:28,824 [output] KeyError:
2013-12-24 06:10:28,898 [output] Command failed, please check log for more info
2013-12-24 06:10:28,911 [output] nova-manage returned exit code 1
2013-12-24 06:10:28,912 [output] Aborting early
2013-12-24 06:10:28,912 [script exit code = 1]

[Michael Still (mikal)]

https://ssl.rcbops.com/turbo_hipster/logviewer/?q=/turbo_hipster/results/56/56729/1/check/gate-real-db-upgrade_nova_mysql_devstack_150/faa4f0e/datasets_devstack_150.log is another example

[Michael Still (mikal)]

Set devstack_150 as non-voting until we sort this one out.

[Daniel Kuffner (daniel-kuffner)]

I think my change set fails becuse of this issue: https://review.openstack.org/#/c/59824/

For example: https://ssl.rcbops.com/turbo_hipster/logviewer/?q=/turbo_hipster/results/59/59824/14/check/gate-real-db-upgrade_nova_mysql_devstack_131007/92efa11/131007_devstack_export.log

What can I do to fix the issue? Or do I have to simply wait until the this issue is closed?
thanks for your help.