Sahara

Migration tests are run regardless of database presence

Bug #1403158 reported by Michael McCune
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Sahara
Fix Released
High
Michael McCune
Sahara 2015.1.0 "kilo"

Bug Description

the database migration tests should only run if a database(mysql, postgres) are detected. these tests are currently being run regardless of database presence.

steps to recreate:

1. clone a fresh sahara repository
2. run the py27 tox tests

the following error is generated:

py27 develop-inst-noop: /home/mike/workspace/testing/sahara
py27 runtests: PYTHONHASHSEED='4028025552'
py27 runtests: commands[0] | python setup.py test --slowest --testr-args=
running test
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover $DISCOVER_DIRECTORY --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover $DISCOVER_DIRECTORY --load-list /tmp/tmphbPCn7
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover $DISCOVER_DIRECTORY --load-list /tmp/tmpZImz3W
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover $DISCOVER_DIRECTORY --load-list /tmp/tmpUpnXt6
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover $DISCOVER_DIRECTORY --load-list /tmp/tmpSHaFnN
======================================================================
FAIL: db.migration.test_migrations.TestMigrationsMySQL.test_models_sync
tags: worker-3
----------------------------------------------------------------------
traceback-1: {{{
Traceback (most recent call last):
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 490, in _cleanup
    with engine.begin() as conn:
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1668, in begin
    conn = self.contextual_connect(close_with_result=close_with_result)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1798, in contextual_connect
    self.pool.connect(),
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 338, in connect
    return _ConnectionFairy._checkout(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 644, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 440, in checkout
    rec = pool._do_get()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 963, in _do_get
    return self._create_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 285, in _create_connection
    return _ConnectionRecord(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 411, in __init__
    self.connection = self.__connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 538, in __connect
    connection = self.__pool._creator()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
    connection_invalidated=invalidated
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (OperationalError) (2002, 'Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (2 "No such file or directory")') None None
}}}

Traceback (most recent call last):
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 594, in test_models_sync
    self.db_sync(self.get_engine())
  File "sahara/tests/unit/db/migration/test_migrations_base.py", line 629, in db_sync
    alembic.command.upgrade(self.ALEMBIC_CONFIG, 'head')
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/command.py", line 165, in upgrade
    script.run_env()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/script.py", line 382, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/util.py", line 241, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "sahara/db/migration/alembic_migrations/env.py", line 95, in <module>
    run_migrations_online()
  File "sahara/db/migration/alembic_migrations/env.py", line 81, in run_migrations_online
    connection = engine.connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1778, in connect
    return self._connection_cls(self, **kwargs)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 60, in __init__
    self.__connection = connection or engine.raw_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1847, in raw_connection
    return self.pool.unique_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 644, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 440, in checkout
    rec = pool._do_get()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1057, in _do_get
    return self._create_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 285, in _create_connection
    return _ConnectionRecord(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 411, in __init__
    self.connection = self.__connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 538, in __connect
    connection = self.__pool._creator()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
    connection_invalidated=invalidated
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (OperationalError) (2002, 'Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (2 "No such file or directory")') None None
======================================================================
FAIL: db.migration.test_migrations.TestMigrationsPostgresql.test_models_sync
tags: worker-0
----------------------------------------------------------------------
traceback-1: {{{
Traceback (most recent call last):
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 490, in _cleanup
    with engine.begin() as conn:
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1668, in begin
    conn = self.contextual_connect(close_with_result=close_with_result)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1798, in contextual_connect
    self.pool.connect(),
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 338, in connect
    return _ConnectionFairy._checkout(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 644, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 440, in checkout
    rec = pool._do_get()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 963, in _do_get
    return self._create_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 285, in _create_connection
    return _ConnectionRecord(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 411, in __init__
    self.connection = self.__connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 538, in __connect
    connection = self.__pool._creator()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
    connection_invalidated=invalidated
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
OperationalError: (OperationalError) could not connect to server: Connection refused
 Is the server running on host "localhost" (127.0.0.1) and accepting
 TCP/IP connections on port 5432?
 None None
}}}

Traceback (most recent call last):
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 594, in test_models_sync
    self.db_sync(self.get_engine())
  File "sahara/tests/unit/db/migration/test_migrations_base.py", line 629, in db_sync
    alembic.command.upgrade(self.ALEMBIC_CONFIG, 'head')
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/command.py", line 165, in upgrade
    script.run_env()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/script.py", line 382, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/util.py", line 241, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/alembic/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "sahara/db/migration/alembic_migrations/env.py", line 95, in <module>
    run_migrations_online()
  File "sahara/db/migration/alembic_migrations/env.py", line 81, in run_migrations_online
    connection = engine.connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1778, in connect
    return self._connection_cls(self, **kwargs)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 60, in __init__
    self.__connection = connection or engine.raw_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1847, in raw_connection
    return self.pool.unique_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 644, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 440, in checkout
    rec = pool._do_get()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1057, in _do_get
    return self._create_connection()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 285, in _create_connection
    return _ConnectionRecord(self)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 411, in __init__
    self.connection = self.__connect()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/pool.py", line 538, in __connect
    connection = self.__pool._creator()
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
    connection_invalidated=invalidated
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/mike/workspace/testing/sahara/.tox/py27/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
OperationalError: (OperationalError) could not connect to server: Connection refused
 Is the server running on host "localhost" (127.0.0.1) and accepting
 TCP/IP connections on port 5432?
 None None
Ran 552 tests in 9.359s (-2.639s)
FAILED (id=1, failures=2, skips=3)
ERROR: InvocationError: '/home/mike/workspace/testing/sahara/.tox/py27/bin/python setup.py test --slowest --testr-args='
___________________________________ summary ____________________________________
ERROR: py27: commands failed

Tags: tests
Michael McCune (mimccune)
Changed in sahara:
assignee: nobody → Michael McCune (mimccune)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to sahara (master)

Fix proposed to branch: master
Review: https://review.openstack.org/142578

Changed in sahara:
status: New → In Progress
Revision history for this message
Viktor Serhieiev (vsergeyev) wrote :

Looks like, that this is not affects common code, sorry for the noise

no longer affects: oslo.db
Sergey Lukjanov (slukjanov)
Changed in sahara:
milestone: none → kilo-2
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (master)

Reviewed: https://review.openstack.org/142578
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=8acf9faf1233081d7c426135c36e83ec8a0e04d7
Submitter: Jenkins
Branch: master

commit 8acf9faf1233081d7c426135c36e83ec8a0e04d7
Author: Michael McCune <email address hidden>
Date: Wed Dec 17 15:56:11 2014 -0500

    Adding database detection to migration tests

    Currently the database migration tests will always run regardless of the
    presence of an active database. This change adds a skip to the models
    sync test if the database is not detected.

    Changes
    * changing the names for the mysql and postgresql detection functions to
      make them more publicly visible
    * adding a function to the test class to gate database presence
    * adding a test_models_sync wrapper to the base migration test class,
      this checks for database presence before running the test

    Change-Id: Ic1c69fffe818763308d831d1ba5c0d030623e0dd
    Closes-Bug: 1403158

Changed in sahara:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in sahara:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in sahara:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.