uuid consistency migration db script failed

[yong sheng gong]

env ubuntu 16.04
gongysh@ubuntu64:/opt/stack/tacker$ dpkg -l | grep -i my
ii libmysqlclient-dev 5.7.12-0ubuntu1.1 amd64 MySQL database development files
ii libmysqlclient20:amd64 5.7.12-0ubuntu1.1 amd64 MySQL database client library
ii mysql-client-5.7 5.7.12-0ubuntu1 amd64 MySQL database client binaries
ii mysql-client-core-5.7 5.7.12-0ubuntu1 amd64 MySQL database core client binaries
ii mysql-common 5.7.12-0ubuntu1 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii mysql-server 5.7.12-0ubuntu1.1 all MySQL database server (metapackage depending on the latest version)
ii mysql-server-5.7 5.7.12-0ubuntu1 amd64 MySQL database server binaries and system database setup
ii mysql-server-core-5.7 5.7.12-0ubuntu1 amd64 MySQL database server binaries
ii netcat-openbsd 1.105-7ubuntu1 amd64 TCP/IP swiss army knife
ii python-mysqldb 1.3.7-1build2 amd64 Python interface to MySQL

2016-07-14 14:27:26.882 | INFO [alembic.runtime.migration] Running upgrade acf941e54075 -> f958f58e5daa, uuid consistency
2016-07-14 14:27:26.926 | Traceback (most recent call last):
2016-07-14 14:27:26.927 | File "/usr/local/bin/tacker-db-manage", line 10, in <module>
2016-07-14 14:27:26.927 | sys.exit(main())
2016-07-14 14:27:26.927 | File "/opt/stack/tacker/tacker/db/migration/cli.py", line 153, in main
2016-07-14 14:27:26.927 | CONF.command.func(config, CONF.command.name)
2016-07-14 14:27:26.928 | File "/opt/stack/tacker/tacker/db/migration/cli.py", line 67, in do_upgrade_downgrade
2016-07-14 14:27:26.928 | do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
2016-07-14 14:27:26.928 | File "/opt/stack/tacker/tacker/db/migration/cli.py", line 45, in do_alembic_command
2016-07-14 14:27:26.928 | getattr(alembic_command, cmd)(config, *args, **kwargs)
2016-07-14 14:27:26.928 | File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 174, in upgrade
2016-07-14 14:27:26.929 | script.run_env()
2016-07-14 14:27:26.929 | File "/usr/local/lib/python2.7/dist-packages/alembic/script/base.py", line 397, in run_env
2016-07-14 14:27:26.929 | util.load_python_file(self.dir, 'env.py')
2016-07-14 14:27:26.929 | File "/usr/local/lib/python2.7/dist-packages/alembic/util/pyfiles.py", line 93, in load_python_file
2016-07-14 14:27:26.930 | module = load_module_py(module_id, path)
2016-07-14 14:27:26.930 | File "/usr/local/lib/python2.7/dist-packages/alembic/util/compat.py", line 79, in load_module_py
2016-07-14 14:27:26.930 | mod = imp.load_source(module_id, path, fp)
2016-07-14 14:27:26.931 | File "/opt/stack/tacker/tacker/db/migration/alembic_migrations/env.py", line 84, in <module>
2016-07-14 14:27:26.931 | run_migrations_online()
2016-07-14 14:27:26.931 | File "/opt/stack/tacker/tacker/db/migration/alembic_migrations/env.py", line 76, in run_migrations_online
2016-07-14 14:27:26.931 | context.run_migrations()
2016-07-14 14:27:26.931 | File "<string>", line 8, in run_migrations
2016-07-14 14:27:26.932 | File "/usr/local/lib/python2.7/dist-packages/alembic/runtime/environment.py", line 797, in run_migrations
2016-07-14 14:27:26.933 | self.get_context().run_migrations(**kw)
2016-07-14 14:27:26.933 | File "/usr/local/lib/python2.7/dist-packages/alembic/runtime/migration.py", line 312, in run_migrations
2016-07-14 14:27:26.933 | step.migration_fn(**kw)
2016-07-14 14:27:26.933 | File "/opt/stack/tacker/tacker/db/migration/alembic_migrations/versions/f958f58e5daa_uuid_consistency.py", line 46, in upgrade
2016-07-14 14:27:26.934 | op.alter_column(table, 'id', type_=types.Uuid)
2016-07-14 14:27:26.934 | File "<string>", line 8, in alter_column
2016-07-14 14:27:26.934 | File "<string>", line 3, in alter_column
2016-07-14 14:27:26.934 | File "/usr/local/lib/python2.7/dist-packages/alembic/operations/ops.py", line 1414, in alter_column
2016-07-14 14:27:26.935 | return operations.invoke(alt)
2016-07-14 14:27:26.935 | File "/usr/local/lib/python2.7/dist-packages/alembic/operations/base.py", line 318, in invoke
2016-07-14 14:27:26.936 | return fn(self, operation)
2016-07-14 14:27:26.936 | File "/usr/local/lib/python2.7/dist-packages/alembic/operations/toimpl.py", line 53, in alter_column
2016-07-14 14:27:26.937 | **operation.kw
2016-07-14 14:27:26.937 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/mysql.py", line 67, in alter_column
2016-07-14 14:27:26.937 | else existing_autoincrement
2016-07-14 14:27:26.938 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 118, in _exec
2016-07-14 14:27:26.939 | return conn.execute(construct, *multiparams, **params)
2016-07-14 14:27:26.940 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-07-14 14:27:26.940 | return meth(self, multiparams, params)
2016-07-14 14:27:26.940 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
2016-07-14 14:27:26.940 | return connection._execute_ddl(self, multiparams, params)
2016-07-14 14:27:26.940 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
2016-07-14 14:27:26.940 | compiled
2016-07-14 14:27:26.940 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2016-07-14 14:27:26.941 | context)
2016-07-14 14:27:26.941 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
2016-07-14 14:27:26.941 | exc_info
2016-07-14 14:27:26.941 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
2016-07-14 14:27:26.941 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
2016-07-14 14:27:26.941 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-07-14 14:27:26.941 | context)
2016-07-14 14:27:26.941 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-07-14 14:27:26.942 | cursor.execute(statement, parameters)
2016-07-14 14:27:26.942 | File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 161, in execute
2016-07-14 14:27:26.942 | result = self._query(query)
2016-07-14 14:27:26.942 | File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 317, in _query
2016-07-14 14:27:26.942 | conn.query(q)
2016-07-14 14:27:26.942 | File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 835, in query
2016-07-14 14:27:26.943 | self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-07-14 14:27:26.943 | File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1019, in _read_query_result
2016-07-14 14:27:26.944 | result.read()
2016-07-14 14:27:26.944 | File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1302, in read
2016-07-14 14:27:26.944 | first_packet = self.connection._read_packet()
2016-07-14 14:27:26.944 | File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 981, in _read_packet
2016-07-14 14:27:26.944 | packet.check_error()
2016-07-14 14:27:26.944 | File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
2016-07-14 14:27:26.944 | err.raise_mysql_exception(self._data)
2016-07-14 14:27:26.945 | File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-07-14 14:27:26.945 | _check_mysql_exception(errinfo)
2016-07-14 14:27:26.945 | File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-07-14 14:27:26.945 | raise errorclass(errno, errorvalue)
2016-07-14 14:27:26.945 | sqlalchemy.exc.DataError: (pymysql.err.DataError) (1171, u'All parts of a PRIMARY KEY must be NOT NULL; if you need NULL in a key, use UNIQUE instead') [SQL: u'ALTER TABLE vims MODIFY id VARCHAR(36) NULL']
2016-07-14 14:27:26.979 | Error on exit

[yong sheng gong (gongysh)]

https://github.com/openstack/requirements/blob/master/global-requirements.txt#L1
alembic>=0.8.4 # MIT

at https://bitbucket.org/zzzeek/alembic/src/8861f49527aef2fe38e54152e79f25ce40707345/alembic/operations/ops.py?at=rel_0_8_4&fileviewer=file-view-default#ops.py-1386

        :param existing_nullable: Optional; the existing nullability
         of the column. Required on MySQL if the existing nullability
         is not being changed; else MySQL sets this to NULL.

so we should use existing_nullable=False so that the alembic will not use 'NULL', to change the values.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/342227

[Sripriya (sseetha)]

additional info here for reference:

this issue is seen on mysql 5.7 version. so any system running this version will see this error. this error is due to the new behavior introduced in 5.7 which fixed some of the old bugs. see below from their release notes.

Columns in a PRIMARY KEY must be NOT NULL, but if declared explicitly as NULL produced no error. Now an error occurs. For example, a statement such as CREATE TABLE t (i INT NULL PRIMARY KEY) is rejected. The same occurs for similar ALTER TABLE statements. (Bug #13995622, Bug #66987, Bug #15967545, Bug #16545198)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/342227
Committed: https://git.openstack.org/cgit/openstack/tacker/commit/?id=5b0492b1a1436af86e193294937f567309c43a1d
Submitter: Jenkins
Branch: master

commit 5b0492b1a1436af86e193294937f567309c43a1d
Author: gong yong sheng <[e-mail]>
Date: Thu Jul 14 22:52:18 2016 +0800

    Don't change existing nullable property

    Regarding why we should do the way in this patch, please see
    bug report.

    exsiting_nullable=False seemingly failed on mysql 5.5,
    so use nullable=False.

    Author: gong yong sheng <[e-mail]>

    Change-Id: Idb7ba72a6273e03f73a28f8b0054ca39ab02c8bc
    Closes-bug: #1603109

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/tacker 0.4.0 release.