execute command ironic-inspector-dbsync on postgre error"NODE_STATE" does not exist

[Miles Fu]

/etc/ironic-inspector/inspector.conf:
[database]
connection = postgresql://username:password@ip:5432/ironic_inspector?keepalives_idle=300&keepalives_interval=30&keepalives_count=5

ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> 578f84f38d, inital_db_schema
INFO [alembic.runtime.migration] Running upgrade 578f84f38d -> d588418040d, Add Rules
INFO [alembic.runtime.migration] Running upgrade d588418040d -> e169a4a81d88, Add invert field to rule condition
INFO [alembic.runtime.migration] Running upgrade e169a4a81d88 -> d2e48801c8ef, Introducing Node.state attribute
Traceback (most recent call last):
  File "/usr/bin/ironic-inspector-dbsync", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/ironic_inspector/dbsync.py", line 94, in main
    CONF.command.func(config, CONF.command.name)
  File "/usr/lib/python2.7/site-packages/ironic_inspector/dbsync.py", line 77, in with_revision
    do_alembic_command(config, cmd, revision)
  File "/usr/lib/python2.7/site-packages/ironic_inspector/dbsync.py", line 82, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/lib/python2.7/site-packages/alembic/script/base.py", line 407, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "/usr/lib/python2.7/site-packages/ironic_inspector/migrations/env.py", line 81, in <module>
    run_migrations_online()
  File "/usr/lib/python2.7/site-packages/ironic_inspector/migrations/env.py", line 76, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/usr/lib/python2.7/site-packages/ironic_inspector/migrations/versions/d2e48801c8ef_introducing_node_state_attribute.py", line 51, in upgrade
    server_default=istate.States.finished))
  File "<string>", line 8, in add_column
  File "<string>", line 3, in add_column
  File "/usr/lib/python2.7/site-packages/alembic/operations/ops.py", line 1535, in add_column
    return operations.invoke(op)
  File "/usr/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/usr/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 123, in add_column
    schema=schema
  File "/usr/lib/python2.7/site-packages/alembic/ddl/impl.py", line 172, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/usr/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
oslo_db.exception.DBError: (psycopg2.ProgrammingError) type "NODE_STATE" does not exist
LINE 1: ALTER TABLE nodes ADD COLUMN state node_state DEFAULT 'finis...
                                           ^
 [SQL: "ALTER TABLE nodes ADD COLUMN state node_state DEFAULT 'finished' NOT NULL"]

[milan k (vetrisko)]

Hi Miles Fu!

Thanks for the report, I've done some googling and found out that with PostgreSQL, folks first create the Enum type to avoid this issues[1][2].

Thanks,
milan

[1] https://stackoverflow.com/questions/37848815/sqlalchemy-postgresql-enum-does-not-create-type-on-db-migrate
[2] https://bitbucket.org/zzzeek/alembic/issues/67/autogenerate-with-enums-on-postgres-dont#comment-4100435
[3] http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#sqlalchemy.dialects.postgresql.ENUM

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/472725

[milan k (vetrisko)]

Miles Fu,

please try applying the patch from the review; I don't currently have a PGSQL env handy...

Cheers,
milan

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/473233

[Miles Fu (fumeng2005)]

https://review.openstack.org/#/c/472725/1
Milan Kováčik already fixed this bug, please assign to him.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Miles Fu ([e-mail]) on branch: master
Review: https://review.openstack.org/473233
Reason: https://review.openstack.org/#/c/472725/1
Milan Kováčik already fixed this bug.

[milan k (vetrisko)]

Miles Fu,

could you please check the patch https://review.openstack.org/#/c/472725 fixes your problem?

Cheers,
milan

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/472725
Committed: https://git.openstack.org/cgit/openstack/ironic-inspector/commit/?id=bc546fc0cdec3e8eac6e6ad900f5c61d28f02710
Submitter: Jenkins
Branch: master

commit bc546fc0cdec3e8eac6e6ad900f5c61d28f02710
Author: dparalen <email address hidden>
Date: Fri Jun 9 16:53:11 2017 +0200

    PGSQL: create Enum before using

    For a reason, Enums in PostgreSQL migrations require explicit
    create[1][2]. This patch changes the introspection state Enum
    definition from a declaration to an explicit construction to
    avoid the issue.

    [1] http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#sqlalchemy.dialects.postgresql.ENUM
    [2] https://bitbucket.org/zzzeek/alembic/issues/67/autogenerate-with-enums-on-postgres-dont#comment-4100435

    Change-Id: I817ce6e42c0ea4ae66369f5569130c7cdc793c83
    Closes-Bug: 1696950

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ironic-inspector 6.0.0 release.