020_migrate_metadata_table_roles fails with "ValueError: Expecting property name"

Bug #1197367 reported by bnight
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Expired
High
Unassigned

Bug Description

Hello All,

After apt-get update && apt-get upgrade keystone failed:

Setting up keystone (1:2013.1.2-0ubuntu2~cloud0) ...
Installing new version of config file /etc/logrotate.d/keystone ...
Installing new version of config file /etc/init/keystone.conf ...

Configuration file `/etc/keystone/keystone.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : start a shell to examine the situation
 The default action is to keep your current version.
*** keystone.conf (Y/I/N/O/D/Z) [default=N] ?
Installing new version of config file /etc/keystone/policy.json ...
Traceback (most recent call last):
  File "/usr/bin/keystone-manage", line 28, in <module>
    cli.main(argv=sys.argv, config_files=config_files)
  File "/usr/lib/python2.7/dist-packages/keystone/cli.py", line 175, in main
    CONF.command.cmd_class.main()
  File "/usr/lib/python2.7/dist-packages/keystone/cli.py", line 54, in main
    driver.db_sync()
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/sql.py", line 156, in db_sync
    migration.db_sync()
  File "/usr/lib/python2.7/dist-packages/keystone/common/sql/migration.py", line 52, in db_sync
    return versioning_api.upgrade(CONF.sql.connection, repo_path, version)
  File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
    return f(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 366, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 91, in runchange
    change.run(self.engine, step)
  File "/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/usr/lib/python2.7/dist-packages/keystone/common/sql/migrate_repo/versions/020_migrate_metadata_table_roles.py", line 30, in upgrade
    data = json.loads(metadata.data)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)
dpkg: error processing keystone (--configure):
 subprocess installed post-installation script returned error exit status 1

I use a mysql database and I login with root user to the db so no auth issues.

My keystone not works now:

(sqlalchemy.engine.base.Engine): 2013-07-03 14:22:03,260 INFO ROLLBACK
(root): 2013-07-03 14:22:03,260 ERROR (OperationalError) (1054, "Unknown column 'token.user_id' in 'field list'") 'SELECT token.id AS token_id, token.expires AS token_expires, token.extra AS token_extra, token.valid AS token_valid, token.user_id AS token_user_id, token.trust_id AS token_trust_id \nFROM token \nWHERE token.valid = %s AND token.id = %s \n LIMIT %s' (1, '64814d7d4765b6bfc47cabb81fa36974', 1)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 236, in __call__
    result = method(context, **params)
  File "/usr/lib/python2.7/dist-packages/keystone/token/controllers.py", line 142, in authenticate
    token_id=token_id)
  File "/usr/lib/python2.7/dist-packages/keystone/common/manager.py", line 47, in _wrapper
    return f(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/keystone/token/backends/sql.py", line 46, in get_token
    token_ref = query.first()
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2156, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2023, in __getitem__
    return list(res)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2227, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2242, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py", line 331, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (OperationalError) (1054, "Unknown column 'token.user_id' in 'field list'") 'SELECT token.id AS token_id, token.expires AS token_expires, token.extra AS token_extra, token.valid AS token_valid, token.user_id AS token_user_id, token.trust_id AS token_trust_id \nFROM token \nWHERE token.valid = %s AND token.id = %s \n LIMIT %s' (1, '64814d7d4765b6bfc47cabb81fa36974', 1)

Please check and let me know howto process with this one.

the version of the keystone package before apt-get upgrade was keystone_2012.1.3+stable-20130423-f48dd0fc-0ubuntu1.1_all.deb

bnight (bnight)
information type: Private Security → Public
Revision history for this message
bnight (bnight) wrote :

Looks like my keystone db is not last version:

DESCRIBE token;
+---------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+-------------+------+-----+---------+-------+
| id | varchar(64) | NO | PRI | NULL | |
| expires | datetime | YES | | NULL | |
| extra | text | YES | | NULL | |
| valid | tinyint(1) | NO | | NULL | |
+---------+-------------+------+-----+---------+-------+

Revision history for this message
bnight (bnight) wrote :

the version of the keystone package before apt-get upgrade was keystone_2012.1.3+stable-20130423-f48dd0fc-0ubuntu1.1_all.deb

description: updated
Revision history for this message
Dolph Mathews (dolph) wrote :

bnight: migration 20 failed while upgrading from essex to grizzly, so your database is not completely in sync with the codebase (hence the subsequent errors).

summary: - keystone-manage db_sync fai,led after keystone upgrade
+ keystone-manage db_sync failed after keystone upgrade
summary: - keystone-manage db_sync failed after keystone upgrade
+ 020_migrate_metadata_table_roles fails with "ValueError: Expecting
+ property name"
Changed in keystone:
status: New → Triaged
importance: Undecided → High
Revision history for this message
bnight (bnight) wrote :

Hi Dolph,

Thank you for this but what I should do in order to fix this ? As I understand the only think that this migration_20 do is to migrate data from metadata table in essex to user_project_metadata in grizzly.

However I have no idea what exactly data this user_project_metadata should have I`m looking at the code at the moment could you please give me an Idea ?

Best Regards,

Hristofor Georgiev

Revision history for this message
Dolph Mathews (dolph) wrote :

Related / may be resolved by bug 1206257

Revision history for this message
Dolph Mathews (dolph) wrote :

I suspect this was fixed at some point, perhaps by the bug linked above? If this is still an issue for anyone, please leave a comment. Thanks!

Changed in keystone:
status: Triaged → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for Keystone because there has been no activity for 60 days.]

Changed in keystone:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.