octane upgrade 7.0 -> 9.0 fails on keystone-manage db_sync

[Alexander J. Maidak]

1) Built Kilo environment with Fuel 7.0 CE
2) Installed octane via yum backed up environment
3) Installed Fuel 9.0 CE
4) Updated 9.0 CE to latest (yum update)
5) Built/Installed octane from stable/mitaka (10.0.0.0b2.dev91)
6) Attempted to restore 7.0 environment via:
 octane fuel-restore --from=/backup-8182016.tar.gz --admin-password=admin

Process failed with:

[2016-08-23 15:42:36,467] INFO octane.util.subprocess puppet[8914] stdout: Debug: Executing 'keystone-manage db_sync'
[2016-08-23 15:42:38,117] INFO octane.util.subprocess puppet[8914] stdout: Notice: /Stage[main]/Keystone::Db::Sync/Exec[keystone-manage db_sync]/returns: 2016-08-23 15:42:37.990 9109 CRITICAL keystone [-] KeyError: <VerNum(56)>

From /var/log/keystone/keystone-manage.log

2016-08-23 15:42:37.990 9109 CRITICAL keystone [-] KeyError: <VerNum(56)>
2016-08-23 15:42:37.990 9109 ERROR keystone Traceback (most recent call last):
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/bin/keystone-manage", line 10, in <module>
2016-08-23 15:42:37.990 9109 ERROR keystone sys.exit(main())
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/keystone/cmd/manage.py", line 46, in main
2016-08-23 15:42:37.990 9109 ERROR keystone cli.main(argv=sys.argv, config_files=config_files)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/keystone/cmd/cli.py", line 992, in main
2016-08-23 15:42:37.990 9109 ERROR keystone CONF.command.cmd_class.main()
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/keystone/cmd/cli.py", line 371, in main
2016-08-23 15:42:37.990 9109 ERROR keystone migration_helpers.sync_database_to_version(extension, version)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/keystone/common/sql/migration_helpers.py", line 210, in sync_database_to_version
2016-08-23 15:42:37.990 9109 ERROR keystone _sync_common_repo(version)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/keystone/common/sql/migration_helpers.py", line 136, in _sync_common_repo
2016-08-23 15:42:37.990 9109 ERROR keystone init_version=init_version, sanity_check=False)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/migration.py", line 79, in db_sync
2016-08-23 15:42:37.990 9109 ERROR keystone migration = versioning_api.upgrade(engine, repository, version)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
2016-08-23 15:42:37.990 9109 ERROR keystone return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2016-08-23 15:42:37.990 9109 ERROR keystone File "<string>", line 2, in _migrate
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
2016-08-23 15:42:37.990 9109 ERROR keystone return f(*a, **kw)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 345, in _migrate
2016-08-23 15:42:37.990 9109 ERROR keystone changeset = schema.changeset(version)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/schema.py", line 82, in changeset
2016-08-23 15:42:37.990 9109 ERROR keystone changeset = self.repository.changeset(database, start_ver, version)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/repository.py", line 225, in changeset
2016-08-23 15:42:37.990 9109 ERROR keystone changes = [self.version(v).script(database, op) for v in versions]
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/repository.py", line 189, in version
2016-08-23 15:42:37.990 9109 ERROR keystone return self.versions.version(*p, **k)
2016-08-23 15:42:37.990 9109 ERROR keystone File "/usr/lib/python2.7/site-packages/migrate/versioning/version.py", line 163, in version
2016-08-23 15:42:37.990 9109 ERROR keystone return self.versions[VerNum(vernum)]

[Ilya Kharin (akscram)]

This case is not supported by the upgrade procedure and seems that the problem is caused by the absence of appropriate migration scripts to support N+2 releases in OpenStack. To support this upgrade path this gap in the migration scripts has to be fixed in MOS Keystone.

[Vitaly Sedelnik (vsedelnik)]

Invalid for MOS per cmment #1 - we don't support upgrade 7.0 to 9.0.

[Alexander J. Maidak (ajmaidak)]

Ok, that makes sense.

An update to the fuel-octane README.md is probably a good idea.

===
Octane - upgrade your Fuel.

Tool set to backup, restore and upgrade Fuel installer and OpenStack environments that it manages. This version of the toolset supports upgrade from versions 7.0 and 8.0 to version 9.1.
===

It would probably better if the last sentence was:

"This version of the toolset supports upgrade from versions 7.0 to 8.0 and 8.0 to version 9.1."